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ABSTRACT 


Title  of  Thesis:  Bayesian  Sequential  Hypothesis  Testing 

David  C.  Mac  Enany,  Master  of  Science,  1986 

Thesis  Directed  by  :  Dr.  John  S.  Baras 

Professor  and  Head  of  the  Systems  Research  Center 
Electrical  Engineering  Department 

In  this  thesis,  optimality  results  are  presented  for  Bayesian  problems  of  sequential  hypothesis 
testing.  Conditions  are  given  which  are  sufficient  to  demonstrate  the  existence  and  optimality 
of  threshold  policies  and  others  are  given  which  help  characterize  these  policies.  The  general 
results  are  applied  to  solve  four  specific  problems  where  the  observations  respectively  arise  from 
a  time-homogeneous  diffusion,  a  progressive  semimartingale  observed  through  a  diffusion,  a  time- 
homogeneous  Poisson  process,  and  a  predictable  semimartingale  observed  through  a  point  process. 
It  is  shown  that  threshold  policies  are  optimal  in  all  four  cases.  Exact  formulas  for  the  Bayesian 
costs  in  the  point  process  cases  will  be  presented  for  the  first  time. 
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Introduction 


This  thesis  addresses  a  class  of  problems  in  the  theory  of  optimal  stopping 
rules,  or  sequential  analysis,  which  are  called  Bayesian  sequential  hypothesis  test¬ 
ing  problems.  In  the  classical  methods  of  mathematical  statistics  one  supposes  that 
an  observation  of  a  stochastic  process  is  available  for  analysis,  but  that  the  length 
of  the  observation  interval  is  fixed  in  advance.  Thus  in  the  classical  binary  hypoth¬ 
esis  testing  problem,  one  fixes  a  time  say  T,  and  then  observes  some  process  say 
{yt  '■  0  <  t  <  T},  whose  statistics  under  each  hypothesis  are  known.  A  decision  is 
then  made  as  to  which  hypothesis  is  the  best  to  choose.  In  contrast  to  this  approach 
wherein  the  length  of  time  that  the  process  is  observed  is  fixed  in  advance,  the  ap¬ 
proach  in  sequential  analysis  is  to  terminate  the  observations  at  a  random  stopping 
time  which  is  chosen  by  the  observer  and  based  upon  the  information  contained  in 
the  data  observed.  In  the  Bayesian  version  of  this  problem,  one  assumes  knowledge  of 
the  prior  statistics  for  each  hypothesis  and  chooses  that  hypothesis  which  best  mini¬ 
mizes  a  chosen  cost  criterion.  The  advantage  of  the  sequential  approach  in  Bayesian 
problems  is  that  one  can  prove  that  they  yield  a  cost  smaller  than  any  other  method 
which  uses  a  fixed  observation  time.  This  advantage  notwithstanding,  in  some  prob¬ 
lems  a  fixed  observation  time  is  inappropriate  due  to  the  nature  of  the  data.  For 
instance,  many  situations  demand  that  a  cost  be  levied  according  to  the  amount  of 
time  it  takes  to  reach  a  decision.  In  particular,  this  is  usually  true  whenever  data 
collection  is  expensive  or  in  situations  where  it  becomes  increasingly  risky  to  continue 


1 


to  observe  the  data.  In  these  situations  one  is  compelled  to  use  a  sequential  method. 

The  principal  goal  of  this  thesis  is  to  demonstrate  the  optimality  of  certain  statis¬ 
tical  procedures,  called  threshold  policies,  as  applied  to  problems  of  Bayesian  sequen¬ 
tial  hypothesis  testing.  The  importance  of  Bayesian  sequential  hypothesis  testing  is 
due  not  only  to  its  applicability  to  many  practical  signal  detection  problems,  but  also 
because  it  provides  a  theoretical  foundation  with  which  to  prove  optimality  results 
for  other  sequential  statistical  procedures. 

The  mathematical  theory  available  to  formulate  and  obtain  optimality  results  in 
sequential  analysis  roughly  divides  into  two  main  categories,  the  first  of  which  can  be 
called  the  ‘dynamic  programming’  approach,  and  the  second  which  might  be  called  the 
‘excessive  function’  approach.  Naturally,  both  approaches  share  many  similarities  but 
are  sufficiently  different  to  permit  this  broad  classification.  The  methodology  adopted 
in  this  thesis  is  best  described  as  falling  into  the  second  category,  but  represents 
a  significant  modification  to  the  usual  development  [DYNKIN],  [SHIRYAYEV  77], 
[THOMPSON]. 

Loosely  speaking,  the  excessive  function  theory  approach  extends  well-known 
results  for  sequences  of  independent  and  identically  distributed  random  variables  to 
continuous-time  (see  also  [KEL’BERT]).  In  this  thesis,  a  more  direct  development  is 
taken,  one  which  yields  a  rich  interplay  between  analytical  and  probabilistic  ideas. 
While  it  is  true  that  that  the  excessive  function  theory  is  more  general  and  hence  more 
powerful,  it  is  also  true  that  the  approach  contained  herein  requires  considerably  less 
sophisticated  mathematical  machinery.  Thus  one  can  argue  that  it  renders  important 
optimality  results  more  accessible  to  a  wider  audience.  Moreover,  although  it  is  less 
generally  applicable,  it  is  nonetheless  powerful  enough  that  with  it  one  can  obtain  the 
solutions  to  four  important  applications,  two  of  which,  to  this  author’s  knowledge, 
have  never  before  appeared. 

The  organization  of  the  thesis  by  chapter  is  from  the  general  to  the  specific,  and 
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is  given  as  follows. 

In  Chapter  I  is  given  a  general,  detailed  treatment  of  certain  types  of  semimartin¬ 
gales,  special  cases  of  which  will  be  needed  later  in  the  applications  chapter.  Also 
given,  is  a  proof  of  the  existence  and  uniqueness  of  the  solution  to  a  functional  differ¬ 
ential  equation.  It  was  a  tough  decision  to  include  this  theorem  in  the  first  chapter, 
but  it  was  felt  that  this  type  of  differential  equation  is  so  intrinsic  to  the  problems 
involving  Poisson-type  processes,  that  relegating  the  result  to  an  unread  appendix 
would  be  cheating  the  reader  out  of  valuable  insight  into  the  nature  of  these  types  of 
problems,  and  their  associated  difficulties.  Indeed,  so  intrinsic  and  problematic  are 
these  equations  that  they  and  their  more  complicated  integro-differential  brethren  are 
deemed  by  Davis  to  be  the  ‘bugbear  of  the  subject’  [DAVIS].  An  interesting  histor¬ 
ical  note  is  that  one  of  the  earliest  (post-Euler)  references  to  equations  of  the  type 
considered  herein  is  due  to  Poisson  himself  [POISSON]. 

In  Chapter  II,  the  general  Bayesian  sequential  hypothesis  testing  problem  is 
discussed  and  formulated.  A  principal  goal  of  the  chapter  is  to  show  how  one  can 
greatly  reduce  the  complexity  of  the  search  for  the  optimal  test.  Also  given  are  initial 
characterizations  of  the  optimal  Bayesian  cost  function. 

In  Chapter  III,  the  principal  optimality  result  is  given,  the  essence  of  which  is  con¬ 
tained  in  Lemma  3.1.1.  Also  included  are  theorems  whose  importance  is  paramount 
in  the  discovery  of  the  unique  test  which  is  optimal  in  a  particular  application. 

In  Chapter  IV,  the  results  of  the  previous  chapters  are  applied  to  solve  four 
Bayesian  sequential  detection  problems,  the  first  involving  a  fully-observed  time- 
homogeneous  diffusion,  and  the  second  involving  a  partially-observed  progressively 
measurable  semimartingale  process.  Partial  optimality  results  for  the  latter  have 
been  previously  presented  only  for  a  special  case  [LaVIGNA].  In  the  third  problem, 
the  Bayesian  sequential  hypothesis  testing  problem  based  on  observations  of  a  Poisson 
process  is  solved.  Explicit  formulae  for  the  risk  and  thresholds  are  presented.  These 
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results  are  then  extended  to  solve  the  last  sequential  detection  problem  wherein  the 
observations  arise  from  a  point  process  whose  underlying  rate  is  one  of  two  predictable 
semimartin  gales . 

After  reading  the  thesis,  one  should  be  able  to  appreciate  the  difficulty  in  handling 
non-diffusion  type  processes  in  sequential  analysis,  especially  in  view  of  the  formulae 
obtained  in  the  point  process  cases.  It  seems  safe  to  say  that  future  work  in  sequential 
analysis  will  deal  with  ever  more  exotic  processes  as  demanded  by  applications,  and 
that  therefore  the  research  in  this  area  will  of  necessity  focus  more  on  approximate 
and  asymptotic  methods.  It  appears  that  a  good  starting  point  for  such  research  is 
with  the  relatively  recent  functional  central  limit  theory  [HELL AND].  In  this  respect, 
the  explicit  formulas  obtained  herein  give  one  a  numerical  yardstick  with  which  to 
judge  candidate  approximation  schemes. 
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Chapter  I 

Prerequisite  Considerations 


1.0  Introduction 

This  chapter  will  establish  some  important  results  which  will  be  used  later  on. 
Each  theorem  will  be  presented  with  slightly  more  generality  than  will  be  necessary 
in  the  hope  that  the  essence  of  each  is  more  clearly  understood. 

Theorem  1.1.1  demonstrates  that  special  types  of  semimartingales  always  escape 
intervals  in  finite  time.  This  result  will  be  used  to  show  that  particular  hypothesis 
testing  strategies — threshold  policies  based  on  these  semimartingales — in  effect  bifur¬ 
cate  the  sample  space  O,  in  the  sense  that  they  terminate  in  finite  time  almost  surely 
under  either  hypothesis. 

Theorem  1.1.2  establishes  a  weak  form  of  the  differential  rule  for  functions  of 
locally  finite  variation  semimartingales  with  piecewise  monotone  sample  paths.  The 
usual  generalized  Ito  differential  rule  is  stated  for  twice  continuously  differentiable 
functions  of  general  semimartingales.  This  is  sometimes  specialized  to  semimartin¬ 
gales  driven  by  discontinuous  martingale  processes  where  the  smoothness  requirement 
is  weakened  to  ‘once  continuously  differentiable’.  It  will  become  necessary  in  later 
chapters  to  consider  stochastic  differentials  of  functions  even  less  smooth,  specifically 
functions  which  are  piecewise  right  continuous  and  have  piecewise  right  continuous 
derivatives.  In  fact,  functions  of  this  type  are  intrinsic  to  the  theory  of  discontinuous 
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semimartingales.  The  reason  is  that  the  formal  application  of  the  usual  differen¬ 
tial  rule  to  discontinuous  semimartingales  very  often  leads  to  functional  differential 
equations  [e.g.  HIBEY,  SEGALL]  of  a  type  whose  solutions  are  generically  not  con¬ 
tinuously  differentiable,  and  which  do  not,  therefore,  satisfy  the  assumptions  which 
led  to  them. 

Theorem  1.2.1  establishes  the  existence  and  uniqueness  of  the  solution  to  a  cer¬ 
tain  functional  advance  differential  equation.  Its  solution  yields  an  important  quan¬ 
tity  arising  in  the  sequential  Bayesian  hypothesis  testing  problem  involving  point 


processes. 
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1.1  Semimartingale  Prerequisites 

Consider  a  probability  triple  (fi,  I,P)  equipped  with  a  right-continuous  filtra¬ 
tion  It,  t  >  0.  In  this  thesis,  the  following  definition  of  semimartingale  will  be  used. 

Definition  1.1.1  A  (P,  /t)-semimartingale  is  a  random  process  X  which  has  the 
decomposition, 

Xt  =  Xq  +  Mt  -f-  At  t  >  0  P-a.s.  (l.l.l) 

where  Xo  is  a  (P,  It)-r.v.,  |J?b|  <  oo  P-a.s.,  M  is  a  corlol  (P,  5rt)-local  martingale, 
with  Mo  =  0,  and  A  is  a  right-continuous,  ^-adapted  process,  initially  zero  and 
of  locally  integrable  variation  [ELLIOTT]. The  value  set  for  X  will  be  denoted 
as  E  C  ]R. 

Without  loss  of  generality  (wlog)  then,  take  the  predictable  version  of  A  and 
note  that  the  random  variable  Aoo  exists  [ELLIOTT].  In  most  of  this  thesis,  the  full 
generality  of  the  definition  will  not  be  needed.  However,  as  stated  previously,  the 
results  given  in  this  chapter  are  somewhat  more  general  than  will  be  necessary  in 
later  chapters  but  that  this  is  done  for  clarity  and  because  the  general  results  were 
no  harder  to  obtain. 

Since  a  principal  theme  of  the  thesis  is  to  consider  threshold  policies  and  hence 
first  exit  times  of  processes  from  intervals,  one  is  naturally  interested  in  the  properties 
that  should  be  satisfied  by  a  process  upon  which  one  intends  to  base  a  threshold 
policy.  One  obvious  requirement  is  that  such  a  process  should  eventually  exit  the 
threshold  interval.  The  next  theorem  gives  three  conditions  which  are  sufficient  to 
guarantee  such  behavior  for  a  semimartingale.  It  is  an  adaptation  and  generalization 
of  Lemma  17.7  in  [LIPTSER  &  SHIRYAYEV  78]. 

Theorem  1.1.1  Let  X  be  a  real  (P,  It) -semimartingale  as  in  definition  1.1.1,  and 
suppose  P{|Aoo|  =  oo}  =  1,  while  P{|At|  =  oo}  =  0,  VO  <  t  <  oo.  Furthermore, 
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suppose  |  E[Af  ]  |  =  E[\Af  |]  for  any  Jt  -stopping  time  f.  Then,  X  will  escape  any 
open  interval  containing  Xq  in  finite  time,  P-a.s. 

Proof: 

Choose  Xq  =  0  P-a.s.  wlog  and  choose  a,  b  to  satisfy  — oo  <  a  <  0  <  6  <  oo, 
a,b  otherwise  arbitrary.  Define  r  =  inf{i  >0  :  Xt  &  (a,  b)}  and  an  =  inf{i  >  0  : 
Jo  |cL4.s|  >  n},n  >  0.  Note  that  r  and  on  are  both  ^-stopping  times  and  thus  so  is 
t  Acrn.  Moreover,  since  A  is  of  locally  integrable  variation,  crn  <  oo  P-a.s.  Vn  >  0, 
and  an  <  crn+i  T  00  P-a.s. 

Next,  assume  that  for  all  n  >  0  it  is  true  that, 

a  <  XrA(rn  <  b.  (1.1.2) 

Otherwise,  there  exists  an  n  >  0  satisfying  r  A  <?„  =  t  <  <rn  <  oo  P-a.s.,  and  such 
that  XTA<Tn  =  XT  £  (a,  6),  from  which  the  theorem  follows.  So  assuming  1.1.2  holds 
for  all  n  >  0,  the  remainder  of  the  proof  will  proceed  by  reductio  ad  absurdum.  From 
1.1.2  then,  using  the  fact  that  M  is  a  (P,  Jt)-local  martingale  one  obtains, 

a  -  b  <  a  <  E[ATA(rn}  <  b  <  b  -  a,  (1.1.3) 

since  clearly  a  —  b  <  a,  and  b  <  b  —  a.  Rewriting  1.1.3  yields, 

|  •Si^rAcT^]  I  <  6-  a  <  oo,  (1.1.4) 

and  since  crn  |  oo  P-a.s.,  then  ArA<Tn  — ►  AT  P-a.s,  and  invoking  the  bounded  conver¬ 
gence  theorem  yields  |P[AT]|  <  b  —  a.  From  this  it  follows  that  P[|a1t|]  <  b  —  a  by 
hypothesis,  and  hence, 

E[\Ar\]  <  oo.  (1.1.5) 


But  then, 


00  >  #[l^r|]  =  £?[1{t=oo}|At|]  +  E[l{  t  <  oo  }\M 

>  ^[l{T=oo}|^|]  =  P7[l{r=0o}|^oo|], 


(1.1.6) 
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and  since  |Aoo|  =  oo  P-a.s.,  it  must  be  true  that  r  <  oo  P-a.s.  That  is  to  say,  X  will 
exit  (a,  b)  almost  surely  in  finite  time.  Since  a  and  b  were  arbitrary  modulo  Xo,  the 
result  is  shown.  I 

Note  that  the  result  can  be  extended  to  a  process  whose  compensator  almost 
surely  takes  on  one  of  two  not  necessarily  infinite  values  at  infinity,  as  long  as  one 
can  exhibit  a  suitably  regular,  bijective  function  which  maps  those  values  into  plus 
and  minus  infinity.  The  theorem  then  holds  for  the  semimartingale  resulting  from  the 
function  acting  on  the  original  process,  and  hence  for  the  original  process  by  inversion. 

Perhaps  it  should  also  be  stressed  that  the  theorem  states  only  a  sufficiency 
result.  For  instance,  it  is  clear  that  no  martingale  can  satisfy  its  hypotheses,  because 
martingales  lack  (nontrivial)  compensators.  On  the  other  hand,  using  the  same  notion 
as  in  the  preceding  paragraph,  a  bijective  function  of  a  martingale  might  yield  a 
process  with  a  nontrivial  compensator  which  indeed  satisfies  the  theorem.  In  fact, 
in  Chapter  IV  there  will  be  a  need  to  guarantee  that  the  exit  times  of  a  certain 
martingale  (an  a  posteriori  probability)  are  almost  surely  finite.  The  approach  taken 
to  accomplish  this  has  just  been  outlined  and  utilizes  Theorem  1.1.1. 

When  a  semimartingale  as  in  definition  1.1.1  escapes  an  open  interval,  questions 
naturally  arise  as  to  its  whereabouts  at  the  time  of  escape.  The  next  definition 
provides  a  handle  with  which  to  phrase  such  questions. 

Definition  1.1.2  Let  X  be  a  real  ( P ,  7t) -semimartingale,  taking  values  in  E  C  ]R, 
and  let  Xo  G  Iq,  P-a.s.,  where  Jo  C  E,  is  some  nonempty  open  interval.  Define  the 
sets, 

fi0  =  {w  G  fl  :  Xq[uj)  G  Jo}) 

U  OO  .=  {W  £  •  -2J<X>(W)  S  Jo}) 

and  suppose  r  is  given  as, 


r  =  inf {<  >0  :  Xt  ^  Jq}. 
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Then  the  A-boundary  of  Iq  is  defined  and  denoted  as, 

dxIo  =  {x  G  E  :  x  =  Xt(w)(<jj)  for  some  u  G  CIq  \  Oqo}. 

In  addition,  the  A-closure  of  Iq  is  defined  and  denoted  as, 

[7o]a  =  lo  U  3xIq . 

It  will  also  be  convenient  to  define  the  upper  and  lower  X-boundary  of  lo  as  <9+ 7o 
and  d~ Iq,  respectively,  i.e.,  dxI0  —  8~Iq  U  <9+7o,  and  x'  G  <9+ 70  implies  x'  >  x  for 
all  x  G  d~ lo,  and  vice  versa. 

Note  that  dxIo  is  P-a.s.  nonempty  if  X  satisfies  the  hypotheses  of  Theorem  1.1.1, 
and  often  dl0  C  dxIo  for  any  such  X,  where  dlo  is  the  usual  boundary  of  70  consisting 
of  its  endpoints.  In  fact,  if  X  is  a  continuous  semimartingale  satisfying  Theorem  1.1.1, 
then  dl0  =  dxIo  P-a.s.  Also  note  that  neither  dxIo  nor  [I0]x  are  necessarily  closed  sets. 
For  instance,  if  X  is  a  real  right  continuous  semimartingale  driven  by  a  discontinuous 
martingale  whose  jumps  are  positive,  then  <9+7o  is  generically  a  semi-open  interval, 
closed  on  the  left  and  open  on  the  right. 

As  it  was  just  indicated,  the  notion  of  the  boundary  of  an  interval  with  respect 
to  a  jump-type  process  is  not  trivial  as  it  is  for  continuous  processes.  The  simplest 
case  is  where  the  jumps  of  the  process  are  always  positive  (negative)  while  the  sample 
paths  are  strictly  nonincreasing  (nondecreasing)  in  between  the  jumps.  Fortunately, 
this  is  the  situation  in  the  binary  hypothesis  testing  of  counting  processes.  It  is  also 
fortunate  because  a  weaker  form  of  the  generalized  Ito  rule  can  be  given  for  a  process 
which  has  such  sample  paths,  and  because  one  soon  discovers  that  the  differential 
rule,  as  it  is  usually  stated,  is  inadequate  to  handle  the  types  of  functions  which 
arise  in  the  hypothesis  testing  of  such  processes.  Specifically,  the  Ito  rule  for  jump 
processes  is  usually  given  for  functions  which  are  once  continuously  differentiable, 
whereas  functions  which  are  less  smooth  need  to  be  considered.  The  next  definition 
gives  a  general  description  of  the  type  of  functions  which  arise. 
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Definition  1.1.3  Let  F  :  E  — *•  JR,  with  E  C  JR,  and  suppose  that  for  all  but  an 
isolated  set  of  real  numbers,  D  C  JR,  F  is  n-times  continuously  differentiable,  n  >  0. 
Further,  at  each  point  xq  £  D  assume  that  limx|Xo  F^(x)  exists  for  k  —  0,  l,,..,n, 
and  define  this  limit  to  be  the  kth  derivative  of  F  at  Xq  G  D.  Also  suppose  that 
limxjXo  FW(x)  exists  and  is  finite  for  all  x0  G  D,  k  =  0, 1,  Then,  F  is  said  to 

be  a  Cn+(E)  function  or  one  writes  F  G  Cn+(E). 

Loosely  then,  a  Cn+  (E)  function  is  a  function  which  is  n-times,  piecewise  right- 
continuously  differentiable,  with  left-hand  limits,  and  only  a  countable  (isolated)  set 
of  points  prevents  it  from  being  n-times  continuously  differentiable.  This  set  of  points 
is  called  the  set  of  breakpoints  of  the  function. 

The  notion  of  monotonic  sample  paths  introduced  above  is  useful'when  consid¬ 
ering  C1+  functions  of  jump  semimartingales.  For  instance,  if  one  applies  a  right- 
discontinuous  function,  such  as  the  greatest  integer  function,  to  a  right-continuous 
jump  semimartingale  with  piecewise  nondecreasing  sample  paths,  then  the  result¬ 
ing  process  will  be  right-continuous.  However,  applying  the  same  function  to  right- 
continuous  semimartingale  with  piecewise  nonincreasing  sample  paths  yields  a  left- 
continuous  process. 

The  next  result  is  an  extension  of  the  usual  differential  rule  to  include  C1+  func¬ 
tions  of  locally  finite  variation  semimartingales  which  have  piecewise  nondecreasing 
sample  paths. 


Theorem  1.1.2  Let  X  be  a  ( P ,  Ft) -semimartingale  of  locally  finite  variation  with 
piecewise  nondecreasing  sample  paths.  Suppose  a  function  F  is  given  such  that 
F  G  C1+(E).  Then  F(X)  is  a  semimartingale,  and  with  equality  denoting  P- 
indistinguishability  [B REMAUD], 

F(Xt)  =  F{X 0)+  /V(Xa_)  dX3+  V]  {AF{X3)-F'{X3-.)AX3},  t  >  0  (1.1.7) 

Jo  0<a<t 


where  AXt  denotes  Xt  —  Xt-  for  all  t  >  0,  any  process  X. 


Chapter  I :  Prerequisite  Considerations 


12 


Proof: 

First  note  that  the  right  continuity  and  left-hand  limits  of  F  are  necessary  in  order 
to  yield  a  corlol  process.  To  see  this,  consider  an  arbitrary  u>  G  0.  Let  t  >  0,  then, 
because  X  is  piecewise  monotone  increasing,  there  exist  {fn}£L0  ,  tn  j  t  such  that 
Xtn(oj)  l  Xt(u>).  But  then  — >•  F(Xt(cj))  because  F  is  right-continuous, 

and  hence  F(X)  is  continuous  on  the  right.  For  emphasis  note  that  in  general, 


limits)  =  F(c- )  ^  F(c)  =  lim F(x)  =  F(c-h) 

xfc  x[c 


(1.1.8) 


It  is  clear  that  F(Xt)  has  limits  on  the  left.  Similarly,  the  right  continuity  of  F'  and 
implied  left-hand  limits  yield  F'(Xa-)  locally  bounded,  and  so  the  integral  in  1.1.7 
makes  sense. 

Let  D  be  the  set  of  disjoint,  consecutive  open  intervals  constructed  from  the 
set  of  breakpoints  of  F,  say  {d*},  so  that  D  =  {...,  (d_i,  d0),  (dQ,  dx),  (dx,  d2), ...}. 
Write  Di  G  D  for  Di  =  (di,d1+1).  Without  loss  of  generality,  suppose  X0  G  D,0 
for  some  Z?,0  G  D.  Define  tq  as  the  first  time  after  t  =  0  that  X  escapes  D,0 
i.e.,  t0  =  inf{t  >  0  :  Xt  £  (di0,di0+i)}.  Now,  as  long  as  t  G  [0,  r0)  ,  the  usual 
differential  rule  [SNYDER]  for  jump  processes  holds  for  F(Xt )  since  F  is  assumed 
strictly  continuously  differentiable  on  £>*„.  Hence  for  t  G  [0,  r0), 


F(Xt)  =  F(X 0)  +  /V(X,_)  dXs  +  V  {A F(X.)  -  F^X.^X,}  (1.1.9) 

J0  0<3<t 

Since  F(Xt)  is  corlol,  then, 


F(XTo-)  =  F(X0)+  r  F'[Xa-)  dXa+  J2  {&F(Xa)—F'(Xa-)AXa}  (1.1.10) 

0  <3<T0- 

Decomposing  dXa  as  dXa  =  dXa  4-  AXa  [WONG]  one  obtains, 

[ro  F'(xa-)dxa=:  [T0 F\xa-)dx:+  T  F'(X3-)AX3 

Jo  Jo  0<3<ro- 


(1.1.11) 
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Combining  1.1.10  and  1.1.11  yields, 

F(XTo-)  =  F(X 0)  +  [T°  F'{Xa-)  dX\  +  Y.  Ai?W  (LL12) 

0  <3<T0- 

Now  the  possibilities  as  to  whether  X  jumps  a  t  =  t0  and/or  F  has  discontinuities 
at  XTo-  and/or  XTo  reduce  to  the  cases  A F(XTo)  =  0  or  A F(XTo)  is  nonzero  but 
bounded  (implicitly  F  is  locally  bounded).  Either  way,  1.1.12  yields, 

F(Xn.)  +  A F(X,0)  =  F(X 0)  +  I"  F'(X.-)  dX‘  +  £  A F(X.)  +  A F(Xr,) 

0  <3<T0- 

(1.1.13) 

and  so, 

F(Xro)  =  F{X0)+  fT°  F'(Xa)dX°a+  Y  AF(X')  (LL14) 

Jo  0<s<t0 

Since  XTo  E  for  some  D{i  E  D  one  can  define  ri  =  inf{£  >  r0  :  Xt  £  I>tl}. 
Then,  if  t  E  [ r0,Ti ),  Xt  E  Dilt  and  F  is  continuously  differentiable  there  so,  arguing 
as  before  it  follows  that, 

F(Xri-)  =  F(Xn)+  T  F'(X,-)dXl+  J2  {*F(X,)-F'(X.-)AX,} 

J Tl  Ti<a<T2  — 

(1.1.15) 

and  from  this, 


F(Xn)  =  F(Xn)  +  ["  F'(X„)  iXZ  +  Y, 

T°  To<3<Tl 

By  induction,  for  any  n  E  IN, 

F(Xr.)  =  F(XTn_,)  +  r  F'(X.-)dXt+  Z  *F(X.) 

''Tn-l  T„_  i  <3<T„ 


(1.1.16) 


(1.1.17) 


where  rn  =  inf{t  >  rn_i  :  Xt  &  D%n},  and  where  3  XTn_x  .  Performing  the 
recursion  indicated  in  1.1.17  one  obtains, 

F(XTn)  =  F(Xo)  +  fn  F\Xa-)dXl+  Y  A*W  (i-1-18) 

Jo  0  <a<r„ 
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Note  that  rn  <  rn+ 1,  and  that  since  X  is  of  locally  finite  variation,  X  can  only  cross 
dD  a  finite  number  of  times  in  finite  time,  P-a.s.  So  if  r  is  any  stopping  time  which 
is  P-a.s.-finite,  there  exists  no  £  IN  such  that  r  A  rno  =  r  P-a.s.,  and  1.1.18  yields, 


F{Xr )  =  F(Xo)  +  [T  F'(Xs-)  dX\  +  J2  A F(Xa)  (1.1.19) 

Jo  0< j<r 


F(Xt)  =  F(X0)  +  fT  F‘{X.J)  dX3  +  Y,  {^F{X3)-F'{X3-)AXs}  (1.1.20) 

0 <3<T 


Since  r  is  any  P-a.s.-finite  stopping  time,  1.1.7  follows  from  1.1.20  a  fortiori.  I 

Hence,  the  usual  differential  rule  in  the  non-diffusion  case  holds  under  slightly 
weaker  conditions  on  the  function  F  than  are  customarily  imposed,  if  one  is  willing 
to  assume  slightly  stronger  conditions  on  the  sample  paths  of  X.  There  are  two  key 
ingredients  to  the  proof.  The  first  is  that  F  must  respect  the  sample  path  properties 
of  X  such  that  F(X)  is  corlol.  For  example,  if  X  has  nondecreasing  sample  paths, 
then  F  must  be  at  least  piecewise  right-continuous.  The  second  ingredient  is  that 
X  must  have  locally  finite  variation.  This  yields  a  boundary  behavior  which  is  suf¬ 
ficiently  simple  to  permit  the  argument  to  proceed  as  in  the  proof.  Without  this 
assumption,  for  instance  in  the  diffusion  case,  this  type  of  argument  is  confounded  at 
the  discontinuities,  where  such  a  process  can  cross  the  boundary  infinitely  often  in  an 
arbitrarily  small  time  period. 

This  concludes  the  general  semimartingale  results  which  will  be  needed  in  the 
applications  chapter.  In  the  next  section,  a  result  is  presented  which  will  be  necessary 
in  the  hypothesis  testing  problem  dealing  with  point  processes. 
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1.2  A  Functional  Differential  Equation 

The  next  theorem  plays  a  key  role  in  Bayesian  hypothesis  testing  problem  in  the 
case  of  point  process  observations. 

Theorem  1.2.1  Let  a,  b,  c,  and  u  be  given  such  that  0  <  a  <  b  <  1,  and  both  c  and 
u  are  positive.  Suppose  a  function  E{x)  is  given  which  is  continuous  for  all  x  G  (0, 1). 
Consider  the  following  functional-advance  differential  equation, 


UR(x)  — -c  xG(0,6),  (1.2.1) 

where  the  linear  operator  U  is  defined  via, 

UR(x)  =  — x(l  -  x)R'(x)  +  (tt  +  a)  |i2(^-U)  -  i?(x)l , 

where  the  boundary  conditions  are  given  by, 

R(a )  =  E(a ) 

u+l  (1-2-2) 

R{x)  =  E(x)  VxG[6,^-^6) 

u  +  b 

and  with  the  functional  requirement, 

R(x )  is  continuous  at  each  x  G  (0,  b).  (1.2.3) 


Then  a  unique  solution  exists  for  all  x  G  (0,6). 

Proof: 

Before  getting  into  the  details  of  the  proof  a  short  informal  discussion  will 
be  given,  since  problems  of  this  type  are  somewhat  peculiar  (see  [EL’SGOL’TS  L 
NORKIN]).  Consider  the  fact  that  as  long  as  x  G  [6  —  ,  b)  that, 


_,.u  +  1  .  _,u  +  1  . 

R[ - x)  =  E{ - x), 

vtt  +  x  '  Ku  +  x  ’ 


(1.2.4) 


and  thus  one  may  write, 


R'(x)  =  F(x,R(x)) 


x  G  Ji[b), 


(1.2.5) 
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where  Ji[6)  is  the  stated  semi-open  interval  and  where, 

(1.2.6) 

with  G(x)  a  known  continuous  function  given  by, 

c+(u  +  x)E{%±±x) 
x{\  -  x) 

Hence,  if  one  can  prove  the  existence  and  uniqueness  of  a  continuous  solution  to 
the  ODE  1.2.5,  with  arbitrary  initial  condition,  then  the  process  may  be  repeated 
inductively  to  the  left,  each  induction  step  involving  a  proof  of  the  existence  and 
uniqueness  of  an  ODE  with  a  different  forcing  function  obtained  from  the  previous 
induction  step.  The  induction  process  is  continued  until  the  ODE  under  consideration 
lives  on  the  interval  containing  x  =  a.  On  this  interval,  call  it  Ja,  the  ODE  solution 
of  interest  is  the  one  passing  through  (a,  E(a)),  which  may  or  may  not  be  an  initial 
condition  for  that  interval.  In  either  case,  assuming  a  unique  trajectory  exists  passing 
through  (a,  E(a)),  then  this  solution  is  extended  to  the  right  hand  endpoint  of  Ja, 
yielding  a  value  for  R  at  this  point.  This  value  of  R  is  then  taken  to  be  the  initial 
condition  for  the  solution  obtained  on  the  previous  induction  step.  Note  that  matching 
the  value  at  this  endpoint  is  justified  since  the  proof  of  the  existence  and  uniqueness 
on  the  previous  step  was  specified  with  an  arbitrary  initial  condition.  The  process 
of  matching  endpoints  is  then  continued  to  the  right  until  the  starting  interval  Jx  [5) 
is  reached,  at  which  time  the  matching  procedure  is  terminated  since  one  is  not  free 
to  choose  the  value  of  E{x)  at  x  =  b.  However,  after  the  induction  step  on  Ja,  and 
after  the  endpoint  matching  procedure,  one  will  have  proven  the  existence  of  a  unique 
continuous  solution  to  the  following  problem, 

UR{x)  =  -e  xe[A,b) 

R(x)  =  E(a ) 

R(x)  =  E{x)  Vx  6  J0  (6), 


(1.2.7) 


(1.2.8) 
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where  A  is  the  left  hand  endpoint  of  Ja,  and  Jo[b)  is  the  semi-open  interval  in  1.2.2. 
The  only  difference  between  1.2.8  and  1.2.1,  is  that  the  solution  to  1.2.1  lives  on  all 
of  (0,6),  whereas  the  solution  to  1.2.8  does  not.  However,  consider  the  problem, 

UR(x)  =  —  c  x  G  (0,  A) 

R(x)  =  R(A+)  (1.2.9) 

R{x)  =  R(x)  Vx  G  Ja • 

If  the  existence  and  uniqueness  of  a  continuous  solution  to  1.2.9  can  be  shown,  then 
by  combining  this  and  the  result  for  1.2.8,  it  follows  that, 

R(x)  —  .R(x)l{x  <  A}  +  .R(x)l{x  >  A},  (1.2.10) 

will  solve  the  original  problem. 

The  problem  1.2.8  will  be  handled  first,  dropping  the  tildes  for  notational  con¬ 
venience.  To  start,  it  is  necessary  to  define  the  discretization  intervals  and  towards 
this  end,  consider  the  advance  E  defined  as, 

E°x  =  x,  Sxx=^ilx,  Snx=  En_1(Sx).  (1.2.11) 

XL  CC 

If  one  asks  that  E-1  satisfy  EE-1  =  E-1E  =  E°,  then  this  yields, 

=  -xT  ,  (1.2.12) 

u  +  1  -  x 

and  E-1  is  called  the  one  step  retardation.  For  any  x  G  (0, 1),  it  follows  easily  that, 

0  <  E-nx  <  x  <  Enx  <1  Vn  >  1,  (1.2.13) 

and  hence  the  significance  of  the  terminology  ‘advance’  and  ‘retardation’.  With  these 
notions,  one  can  define, 


Jn[b)  =  [E-"6,E-(Tl-1)6)  n  >  0, 


(1.2.14) 
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as  the  nth  discretization  interval;  this  agrees  with  the  previous  definition  given  for 
Jo[b)  given  above.  Note  that  there  will  be  occasion  to  use  the  notation  Jn(x),  with  the 
obvious  interpretation.  Observe  that  E~nx  (Enx)  is  monotone  decreasing  (increasing) 
and  bounded  below  (above).  Hence,  there  exists  n  =  na  such  that  E-na5  <  a,  and 
such  that  this  is  the  smallest  such  number  satisfying  the  inequality.  In  the  previous 
informal  notation  then,  Ja  =  Jno[6),  and  A  —  E-n<,6.  Thus,  the  existence  and 
uniqueness  of  a  soution  to  1.2.8  will  follow  if  one  can  prove  this  for  solutions  of 

R'n(x)  =  Fn(x,  Rn(x))  xeJn [6) 

(1.2.15) 

i2n(S"n6)  =  Rn, 

for  each  1  <  n  <  na ,  and  in  addition  do  the  same  for  a  solution  of, 

^na  (x)  =  Fna  (a;,  Rna  (x))  X  E  Jna  (6) 

(1.2.16) 

Rna  =  E(a), 

where  in  1.2.15,  Rn  is  an  arbitrary  constant  and, 

F„(x,R)  =  G4x)-^±^R,  (17) 


where, 


with, 


Gn(x)  = 


C+{u  +  x)Rn-1 


x(l  -  x) 


Rq(x)  =  E(x)  x&J0[b). 


(1.2.18) 


(1.2.19) 


Let  n=l,  for  the  first  induction  step.  Then  it  is  a  simple  consequence  of  elementary 
ODE  theory  that  a  unique  solution  to  1.2.15  exists  passing  through  any  point  interior 
to  the  set  (Ji[6),]R).  To  prove  a  unique  solution  exists  starting  from  the  boundary 
point  (E_16,Il),  one  can  choose  ei  >  0  such  that  that  E-16  —  >  0,  and  then  define 

Ji(b)  =  (E-16  —  ei, 6).  Using  these  open  intervals,  consider  the  auxiliary  ODE, 

R'(x)  =  Fi(x,  R(x))  x  E  Ji(b) 


i2(E-16)  =  Rx. 


(1.2.20) 
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Now  apply  ODE  theory  to  1.2.20.  Specifically,  Fi(x,  R)  is  continuous  for  all 
(x,R)  in  the  open  set  (Ji(6),R).  In  view  of  1.2.19,  Fi  has  a  continuous  first  partial 
derivative  in  R  for  all  (x,R)  G  (Ji(6),lE).  So,  applying  the  standard  results  (see 
HALE,  Theorem  3.1),  R  satisfying  1.2.20  exists  and  is  unique.  Therefore,  if  one 
defines  the  restriction, 

Ri(x)  =  R(x)  zgJi[6),  (1.2.21) 

then  Rn(x)  uniquely  exists  satisfying  1.1.15  for  n  =  1.  Notice  from  1.2.15  that  J?((x) 
is  continuously  differentiable  for  all  x  G  Jn(6).  Therefore,  Ri(x)  and  thence  also 
G2{x)  is  continuous  there.  The  proof  for  any  n  <  na  is  the  same  as  for  n  =  1  with 
the  following  observations.  First,  realize  that  one  can  always  find  an  en,  n  >  1, 
satisfying  E-n6  —  en  >  0,  in  view  of  1.2.13.  Thus  for  any  n  >  1,  Jn(b)  is  well-defined. 
Second,  from  1.2.17,  it  is  clear  that  Fn(x,  R)  has  a  continuous  first  partial  in  R  for  all 
[x,  R)  G  ((0, 1),  1R),  and  hence  this  is  true  for  all  (re,  R)  G  (Jn(5),  1 R).  Lastly,  it  should 
be  clear  that  Fn(x,R)  is  continuous  for  all  (x,R)  G  (Jn(6),lR).  It  is  clearly  true  for 
n=l,  has  been  shown  to  be  true  for  n  =  2,  and  is  true  by  induction  in  general.  In 
summary,  a  unique  solution  to  1.2.15  with  arbitrary  initial  condition  has  been  shown 
to  exist  for  each  n,  1  <  n  <  na.  The  proof  of  the  existence  of  a  unique  solution 
to  1.2.16  is  the  analogous.  If  a  =  E~n“b,  then  the  proof  is  exactly  the  same.  If 
a  >  E then  there  is  no  need  to  consider  the  auxiliary  problem  since  then  ‘a’  is 
interior  to  JUa[b ),  and  the  proof  follows  directly. 

To  begin  the  endpoint  matching  process,  note  that  since  Fn  is  so  well  be¬ 
haved  for  all  (x,R)  G  ((0,1),  1R),  then  the  solution  on  Jna[b)  can  be  extended  nat¬ 
urally  to  include  the  right-hand  endpoint  (see  HALE,  Lemma  2.1).  Then  define 
i2na_i(E-(n“-1)6)  =  Rna  (E-^na_1^6).  Continue  this  process  of  extension  and  match¬ 
ing  up  to  and  including  J2[& )•  The  unique  solution  to  1.2.8,  for  a,  b  as  given,  has  now 
been  demonstrated  to  exist. 

To  complete  the  proof,  it  is  necessary  to  extend  the  solution  continuously  to  the 
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left,  i.e.,  solve  the  problem  given  in  1.2.9.  First  observe  that  the  solution  to  1.2.8 
can  be  extended  naturally  to  the  left-hand  endpoint  of  Jna  [6)  (  again,  see  HALE, 
Lemma  2.1  ).  Thus  solving  1.2.9  is  equivalent  to  proving  the  existence  and  uniqueness 
of  a  solution  to  (hats  removed  for  notational  politeness), 

R'n(x)  =  Fn(x,Rn(x))  x  e  J„[6) 


ij,n((S_(n_1)6)_)  =  i2n, 


(1.2.22) 


for  all  n  >  na,  with  Rna+ 1  =  Rna  (E-na6).  The  proof  in  this  case  is  also  by  in¬ 
duction  on  the  analogous  ‘over barred*  problem  (see  1.2.20),  where  now  Jn(6)  = 
(E-n6,  E-(n-1)&-|-en),  and  again,  en  >  0  can  always  be  chosen  so  that  E  (n  1^6+ cn  < 
1.  This  time,  at  each  step  of  the  induction,  the  solution  is  extended  naturally  to  the 
left,  providing  an  initial  condition  on  the  next  induction  step.  Finally,  since  for  any 
x  £  (0,6),  there  exists  a  smallest  nx  such  that  E-rix  <  x,  the  solution  to  1.2.9  is 
secured.  Recalling  1.2.10,  the  theorem  has  been  shown.  I 

Corollary  If  in  addition,  E(x)  is  n-times  continuously  differentiable  on  J0[6),  n  >  1, 
then  the  unique  solution  to  the  problem  is  also  n-times  continuously  differentiable 
except  possibly  for  x  £  {E-fe6  :  k  =  0,  ...n}.  Also,  if  R{b~)  =  R( 6)  for  b  given,  then 
R(x)  is  n-times  continuously  differentiable  for  all  n  >  1,  and  x  <  b. 

Proof:  Follows  directly  from  1.2.1;  solve  for  the  derivative  and  use  induction.  I 
This  concludes  the  current  section  and  chapter.  In  the  next  chapter,  the  problem 
of  Bayesian  sequential  hypothesis  testing  is  presented. 


Chapter  II 

Bayesian  Sequential  Hypothesis  Tests 


2.0  Introduction 

In  this  chapter,  the  Bayesian  sequential  hypothesis  testing  problem  is  presented.  The 
first  section  establishes  the  probabilistic  framework  which  will  be  used  throughout  the 
thesis.  In  addition,  the  class  of  possible  solutions  to  the  sequential  hypothesis  testing 
problem  is  defined.  Within  this  class  of  so  called  admissible  policies,  is  the  class  of 
threshold  policies  which  is  also  defined.  The  second  section  gives  the  particulars  of 
the  general  Bayesian  problem,  however,  only  the  binary  case  is  presented.  The  Bayes 
risk  is  defined  and  is  shown  to  be  concave. 

In  section  2.3,  the  first  steps  are  taken  to  reduce  the  complexity  of  the  mini¬ 
mization  problem  posed  by  the  Bayes  risk.  Theorem  2.3.1  shows  that  for  any  policy 
u  =  (?,  «5)  £  U,  one  can  replace  6  with  a  certain  /^-measurable  random  variable  and 
by  so  doing  obtain  a  new  policy  whose  risk  is  not  larger.  The  immediate  consequence 
is  that  the  infimum  over  all  admissible  policies  is  no  smaller  than  the  infimum  over 
their  first  component.  Further  insight  is  gained  in  Theorem  2.3.2  which  helps  charac¬ 
terize  the  form  of  the  risk.  Next  it  is  shown  that  threshold  policies  are  unchanged  by 
the  replacement  procedure  of  Theorem  2.3.1  —  an  obvious  property  to  check  before 
attempting  to  prove  that  such  policies  are  optimal.  The  chapter  ends  by  demonstrat¬ 
ing  that  policies  invariant  under  Theorem  2.3.1,  and  in  particular  threshold  policies, 
have  terminal  errors  whose  sum  is  bounded  by  one. 
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2.1  Preliminaries 

In  this  section,  the  problem  of  testing  two  statistical  hypotheses  will  be  presented 
using  the  Bayesian  risk  approach.  The  following  probablistic  framework  forms  the 
starting  point  of  the  investigation. 

On  a  measurable  space  (f2,  J),  there  are  given  two  additional  probability  mea¬ 
sures  Pi  ,  i=0,l.  One  is  given  a  filtration  {Ot  :  t  E  T},  where  T  C  [0,  oo)  is  the 
time  parameter  set,  either  continuous  or  discrete.  If  T  =  [0,  oo),  then  Ot  is  taken  to 
be  right  continuous.  The  Ot  filtration  should  be  thought  of  as  being  the  history  of 
some  observable  process.  There  is  also  given  0,  the  hypothesis  parameter  set.  For 
instance,  in  the  simplest  case  of  binary  hypotheses,  ©  =  {0q,0i}. 

The  general  hypothesis  testing  problem  can  be  loosely  described  as  discovering 
a  plan,  or  policy,  which  dictates  when  to  stop  and  what  to  decide,  based  upon  the 
available  information,  and  which  also  satisfies  a  given  performance  criterion.  Any 
policy  then,  ought  to  be  a  pair  consisting  of  a  stopping  time  and  a  deciding  rule,  and 
an  optimum  policy  is  one  which  achieves  a  specified  performance.  In  view  of  this,  it  is 
clear  that  in  seeking  optimum  policies,  the  search  is  naturally  limited  to  a  class  of  can¬ 
didate  policies  whose  membership  in  the  class,  or  admissibility,  is  defined  by  practical 
measurability  requirements.  That  is  to  say,  each  component  of  an  eligible  policy  pair 
should  at  least,  and  at  most,  be  measurable  with  respect  to  some  c-algebra  which 
contains  all  the  information  that  might  possibly  be  observed.  A  natural  definition  of 
admissible  policy  then,  is  given  in  terms  of  the  observation  filtration,  Ot  ■ 

Definition  2.1.1  An  admissible  policy,  u,  is  a  pair,  u  —  (r,  5),  where  r  is  an  0t- 
stopping  time  taking  values  in  T,  and  6  is  a  ( P ,  0T)-random  variable  taking  values  in 
0.  The  class  of  admissible  policies  will  be  denoted  as  U. 

In  the  binary  hypothesis  testing  problem,  the  choice  of  u  E  U  prescribes  a  policy 
which  specifies,  via  r,  when  the  observation  is  to  be  terminated,  and  specifies  via  6, 
which  hypothesis  is  to  be  accepted.  That  is  to  say  if  S  =  i,  then  the  hypothesis  0  =  0, 
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will  be  accepted,  *  =  0, 1. 

An  important  subclass  of  admissible  policies  are  the  threshold  policies.  This  is 
due  both  to  their  simple  specification  and  remarkable  optimality  properties. 

Definition  2.1.2  Let  X  be  an  Ot-adapted,  TR-valued  process.  An  (AT,  7)-thresliold 
policy  u  G  U  is  a  pair,  u  =  (f,  6),  where, 


¥  =  inf{£  >  0  :  Xt  ^  1} 


and, 


Xf  >  b 
Xf  <  a 


where  a  <  b  are  the  thresholds,  and  I  —  (a,  b)  is  called  the  threshold,  or  con¬ 
tinuation  interval.  The  class  of  threshold  policies  will  be  denoted  as  U;  note  that, 
U  C  U.  If  a  =  b,  the  threshold  policy  is  said  to  be  degenerate  and  one  defines  f  =  0 
and  6  —  1  if  Xo  —  b. 

Observe  that  for  a  given  filtration,  there  are  as  many  possible  threshold  policies 
as  there  are  ways  to  choose  a  pair  (X,  I )  which  satisfies  the  definition.  For  a  given 
Ot-adapted  process  X  however,  the  hope  is  that  one  can  find  a  priori  a  particular 
threshold  pair  such  that  the  threshold  policy  so  defined  is  optimal  in  some  sense. 
For  our  purposes,  X  will  be  taken  to  be  either  a  certain  a  posteriori  probability 
or  (log) likelihood  ratio  process,  the  important  similarity  being  that  they  all  satisfy 
Theorem  1.1.1.  As  a  matter  of  style,  the  policies  in  Definition  2.1.2  will  be  called 
simply  ‘threshold  policies’  when  there  is  no  ambiguity  in  either  the  reference  process 
or  threshold  interval. 

Consider  an  arbitrary  admissible  policy  u  G  U.  For  each  u>  €  0,  the  policy 
u  =  (r,  6 )  can  be  described  very  generally  as  incurring  two  kinds  of  losses,  one  due 
to  the  cost  of  waiting  to  decide,  and  the  other  due  to  making  a  wrong  decision.  A 
reasonably  general  characterization  of  the  cost  of  the  observation  time,  or  running 
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cost,  is  given  by  ca  ds,  where  {ct ,  t  >  0}  is  some  0*-adapted  process  which  serves 
as  a  suitable  cost  measure.  For  instance,  c<  =  c  >  0,  a  positive  constant,  is  often 
chosen  with  the  intention  of  using  an  apparently  simple  running  cost  to  capture  the 
behavior:  ‘the  longer  it  takes  to  decide,  the  more  it  costs’.  Of  course,  this  is  not 
the  only  cost  process  which  captures  this  behavior,  nor  is  it  necessarily  simple,  for 
the  simplicity  of  a  running  cost  is  often  best  judged  not  by  its  specification,  but  by 
how  easily  it  yields  to  mathematical  analysis.  Indeed,  a  fundamental  tradeoff  in  the 
modeling  aspects  of  a  sequential  testing  problem  is  choosing  a  running  cost  which  can 
be  worked  with  mathematically,  and  still  captures  a  desired  behavior. 

Next,  consider  that  each  admissible  policy  can  make  two  types  of  incorrect  deci¬ 
sions, 

{6  =  0  while  0  ~  9\ 

6  =  1  while  6  =  9q. 

Naturally,  it  will  be  desirable  to  include  in  the  total  cost  structure  a  penalty  for 
each  of  these  events  and  then  minimize  in  some  way  the  total  cost,  and  in  this  sense 
minimize  the  probabilities  of  these  errors. 

This  concludes  the  section.  In  the  next,  the  discussion  will  focus  on  the  details 


of  the  Bayesian  formulation. 
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2.2  Bayesian  Formulation 

Let  9  be  a  {0,  l}-valued,  /-measurable  random  variable,  0:0—*©.  Fix  an 
arbitrary  n  €  [0, 1]  and  define  a  probability  measure  Pn  on  (ft,  7)  via, 

P„{u  :  0(w)  —  1}  =  7r  ,  P„{<jJ  :  9(w)  =  0}  =  1  -  ir,  (2.2.1) 

such  that  for  each  set  A  &  7  there  holds, 

1  P,{A)  =  nPx{A}  +  (1  -  n)P0{A}.  (2.2.2) 

In  words,  it  is  assumed  that  9  is  a  binary  random  variable  taking  on  the  values  zero 
or  one  with  the  a  priori  probabilities  1  —  ir  and  7T,  respectively.  The  a  posteriori 
probability  of  the  0  =  1  hypothesis  is  then  defined  as  wt  —  Pv{9  =  l|0t},  t  &T,  and 
since  Oo  =  {0,  ft},  it  follows  that  Pw{7To  =  7r}  =  1. 

Consider  an  arbitrary  admissible  policy  u  6  U  as  defined  above.  For  each  ui  E  ft, 
the  policy  u  —  (r,  S)  can  be  described  as  incurring  two  kinds  of  losses,  one  due  to 
the  cost  of  waiting  to  decide,  the  other  due  to  making  wrong  decisions.  On  average 
therefore,  a  natural  definition  for  the  average  running  cost  is  given  by  -EWf/o  cads], 
where  { ct,t  >  0}  is  some  nonnegative  0*-adapted  process  which  yields  a  desired  cost 
behavior. 

To  describe  the  cost  of  making  wrong  decisions,  define 

c°  if  9  =  1,  6  =  0;  c°,  >  0 
c1  if  0  =  0,  $=1;  c1,  >  0  (2.2.3) 

0  if  6  =  9, 

yielding  the  average  cost  of  incorrect  decisions  as  E7r[w{9,6)\.  Observe  that  it  is 
without  loss  of  generality  in  the  search  for  an  optimal  policy  that  there  is  no  cost 
levied  for  correct  decisions  in  2.2.3.  Now,  one  can  easily  show  that 


Ev[w{9,8)\  =  c°t vPx{6  =  0}  +  cx(l  -  t t)P0{6  =  1}. 
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Putting  the  pieces  together  yields, 

p(it,u)=E„[f  c3ds  +  u>(0,<$)]  (2.2.4) 

Jo 

as  the  total  P„- average  Bayesian  cost,  or  risk,  of  the  policy  u  =  (r,S).  By  definition 
it  is  clear  that  p(i r.-u)  is  nonnegative  for  all  it  £  [0, 1]  and  u  £  U,  and  therefore  the 
Bayes  risk,  or  cost,  given  by, 

p(it)  =  inf  p(it,  u)  (2.2.5) 

is  nonnegative  and  exists  for  all  it  £  [0, 1].  For  a  particular  it  £  [0, 1],  one  would  like 
to  find  a  u  £  U  which  minimizes  the  risk  at  that  prior.  From  2.2.5  it  is  easy  to  show 
that 

P(0)  =  0  =  />(l),  (2.2.6) 

and  thus  for  it  £  {0,  l},  an  optimal  policy  is  u  =  (r,  6)  =  (0, 7r),  i.e.,  stop  immediately 
and  decide  it.  Obviously  this  trivial  policy  is  not  optimal  for  other  values  of  it,  but  any 
rival  policy  which  is  must  behave  as  u  for  it  £  (0, 1},  i.e.,  it  must  stop  immediately 
and  decide  it. 

From  2.2.5  it  also  follows  that  p(it)  is  concave.  This  important  observation  is 
recorded  in  the  following  theorem  for  future  reference. 


Theorem  2.2.1  The  Bayes  risk  (2.2.5)  is  concave  in  it  on  [0, 1], 


Proof: 

Combine  2.2. 1-2. 2. 4  to  get, 

p(it,u)  =  Ei[  f  c„  ds  +  cQitTl{8  =  0}]  -  E0[  f  ca  ds  +  cx(l  -  7rT)l{<S  =  1}] 
Jo  Jo 

+  E0[J  cs  ds  +  cx(l  -  7rT)l{<$  =  1}]. 

With  this  in  mind,  it  follows  that  for  any  a  £  (0, 1),  and  it',  it"  £  [0,  l], 
p(ocit'  +  (1  —  a)7r")  =  inf  p(ait'  -I-  (1  —  a)it",u) 

>  a  inf  pfit'.u)  +  (l  -  a)  inf  p(ir",u) 

=  ap{it')  +  (1  -  a)p(it"), 


it 
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The  desired  result.  I 

The  following  definition  serves  to  characterize  the  optimality  of  admissible  poli¬ 
cies  for  the  Bayesian  formulation. 

Definition  2.2.1  Let  I  C  [0, 1].  The  policy  u*  G  U  is  said  to  he  /-Bayesian  if 

p(n,u*)  =  p(ir)  Vtt  G I  (2.2.7) 

If  u*  GU  is  [0, 1] -Bayesian,  it  is  said  to  be  Bayesian. 

Consider  that  any  reasonable,  admissible  policy  will  be  at  least  {0, 1}-Bayesian, 
and  so  for  convenience  of  notation,  /  will  be  understood  to  denote  I  U  {0, 1}  unless 
noted  otherwise.  Given  this  definition,  one  of  the  principal  goals  of  this  thesis  can 
be  succinctly  stated  as  demonstrating  that  there  exists  a  threshold  policy,  say  with 
thresholds  a,  b,  a  <  b,  which  is  at  least  (a,  6)-Bayesian,  and  preferably  Bayesian,  for 
a  given  observation  filtration.  It  is  this  goal  of  demonstrating  the  existence  of  such  a 
threshold  policy  and  thresholding  interval  which  motivates  the  use  of  the  suggestive 
‘F  notation  in  the  definition. 

For  completeness,  one  technical  assumption  perhaps  deserves  mention.  In  2.2.3 
it  is  assumed  that  both  costs  for  wrong  decisions  are  strictly  positive.  This  is  done  to 
avoid  degenerate  cases  for  which  no  Bayesian  threshold  policy  exists  or  is  unique.  For 
instance,  let  i  G  {0, 1}  and  suppose  cx  —  0,  while  the  other  cost  is  nonzero.  Then  it 
is  easy  to  show  that  the  policy  ut-  =  (r,  Si)  =  (0,i)  is  ([0,  l]-)  Bayesian.  It  also  follows 
that  there  exists  no  threshold  policy  which  is  Bayesian  optimal  (consider  -k  —  \i  —  l|). 
Likewise,  if  c°  =  c1  =  0,  then  any  policy  with  stopping  time  r  =  0  P^-a.s.  Vtt  G  [0, 1] 
is  Bayesian,  and  this  includes  all  the  degenerate  threshold  policies. 

This  concludes  the  section.  The  next  section  begins  by  showing  how  the  infimum 
over  all  admissible  policies  appearing  in  2.2.5  can  be  replaced  by  a  much  simpler 
minimization.  This  result  is  the  first  to  reveal  why  threshold  policies  enjoy  their 
special  status. 
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2.3  Optimal  Stopping 

Having  considered  basic  notions  of  Bayesian  hypothesis  testing  in  the  previous  sec¬ 
tions,  the  focus  here  is  to  lessen  the  complexity  of  the  search  for  the  optimal  policy. 
Towards  this  end  it  is  shown  in  Theorem  2.3.1  that  finding  a  Bayesian  policy  can 
be  reduced  to  a  problem  of  optimal  stopping,  i.e.,  one  can  restrict  the  search  for  the 
optimal  policy,  an  infimum  operation  over  both  policy  components,  to  a  search  for 
an  optimal  stopping  policy,  an  infimum  operation  over  the  stopping  time  component 
alone.  This  is  accomplished  by  showing  that  any  policy  pair  can  be  replaced  by  an¬ 
other  pair  whose  risk  is  no  larger.  The  new  pair  is  obtained  from  the  original  by 
replacing  its  decision  component  with  a  new  decision  variable  which  has  an  explicit 
representation  in  terms  of  the  a  posteriori  probability;  the  stopping  time  component 
is  kept  the  same.  This  yields  a  simpler  expression  for  the  Bayes  risk  which  is  then 
exploited  to  discover  more  of  its  structure.  Also,  the  question  arises  as  to  the  effect 
of  the  replacement  procedure  in  Theorem  2.3.1  on  threshold  policies,  and  it  is  shown 
that  such  policies  remain  unchanged — an  obvious  property  to  check  before  one  tries 
to  prove  that  threshold  policies  are  Bayesian!  Prior  to  this,  the  theorem  is  used  to 
further  characterize  the  risk  as  was  begun  in  Theorem  2.2.1.  The  section  ends  by 
showing  that  the  sum  of  the  terminal  errors  for  a  threshold  policy  are  bounded  by 
one.  The  next  theorem  is  a  generalization  of  Lemma  4.1.1  in  [SHIRYAYEV]. 

Theorem  2.3.1  Let  u  —  (t,6)  E  U  be  arbitrary,  and  define  u  =  (r,  6)  E  U  via,  t  =  f 


and, 

(  1  C°7Tt  >  c1  (1  —  7Tt) 

6  = 

(2.3.1) 

[  0  c°7tt  <  cx(l  —  7 rT). 

Then,  for  all 

7T€  [0,  1], 

p(TT,u)>p(ir,u)  =  En[  c3ds  +  e(  ttt)] 

(2.3.2) 

Jo 

where, 

e(n)  =  min{c°7r,  c1  (1  —  tt)}, 

(2.3.3) 
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and  {ct  :  t  >  0}  is  an  Ot-adapted,  nonnegative  process. 

Proof:  If  7r  =  0  or  1,  then  the  assertions  are  easy  to  check,  so  suppose  7r  £  (0, 1). 
From  definition  2.2.1,  it  is  clear  that  since  {£  =  i}  £  Of  for  i  =  0, 1,  there  holds, 


Ev[l{6  =  O}1{0  =  1}]  =  Er[  1{6  =  (>}£?.  [l{*  =  1>|  0f]] 

=  Ew[l{6  =  0}7Tf]. 

and  similarly, 

E«{\{6  =  1}1{0  =  0}]  =  E,[l{6  =  1}(1  -  7rf)]. 


(2.3.4) 


(2.3.5) 


So,  combining  2.2.3,  2.2.4,  2.3.4,  and  2.3.5  yields, 


p(n,u)  =  E„[ 

[  c3ds  +  w(0,6)] 

1 0 

=  En[ 

w 

f  Cgds  +  c°l{6  =  O}l{0  =  1}  + 
'o 

c1 l{6  =  l}l{0  =  0}] 

f  cads  4-  c°(l  —  6)iVf  +  0*6(1  — 
'0 

**)] 

=  E«[ 

I  c3ds  +  c°(l  —  6)7Tt  +  c*6(l — 
'o 

*t)\ 

>EV[ 

/  Cgds  +  e(7TT)]  =  p{tt,u), 

'  0 

and  the  theorem  is  shown.  I 

Observe  that  the  theorem  is  very  general.  No  assumptions  were  made  regarding 
the  continuity  of  either  the  time  parameter  or  state-space;  one  has  only  to  interpret 
the  integrals  in  the  proper  way.  The  extension  to  multiple  hypotheses  is  tedious 
but  straightforward.  In  addition,  notice  from  definition  2.1.2  that  a  threshold  policy 
(f,S)  £  U  is  entirely  specified  by  its  first  component  (via  the  thresholds).  That 
is  to  say,  if  f  is  given,  6  is  completely  determined,  a  characteristic  not  shared  by 
admissible  policies  in  general.  The  theorem  however  shows  that  in  minimizing  the 
risk,  the  infimum  over  all  policies  is  no  smaller  than  the  infimum  over  all  policies 
which  share  this  characteristic.  This  is  a  welcome  simplification  of  the  problem  for  it 
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says  that  in  the  search  for  Bayesian  decision  policies,  one  need  only  consider  policies 
u  =  (r,  £)  with  6  given  by  2.3.1  and  with  the  risk  given  by, 

p(ir)  =  inf  p(w,u)  =  inf  p{n,r), 


where, 

p(r,r)  =  E„[f  c3ds  + e(7rT)],  (2.3.7) 

Jo 

and  where  T  denotes  the  class  of  Ot-stopping  times.  In  fact,  without  loss  of  generality 
it  is  sufficient  to  take  T  to  be  the  subclass  of  P„- a.s.  finite  stopping  times  if  the 
following  technical  condition  holds, 

r  oo 

Er[  cads]  =  oo. 

Jo 

Intuitively,  the  condition  (T)  is  entirely  natural  since  it  is  unusual  to  impose  a  running 
cost  which  fails  to  force  decisions  in  finite  time  in  problems  of  sequential  hypothesis 
testing.  As  a  result,  it  will  be  assumed  throughout  the  remainder  of  this  thesis  that 
condition  (T)  holds.  As  a  sidenote  the  reader  should  be  aware  that  the  cost  due  to 
making  wrong  decisions  is  often  called  the  ‘terminal  cost’  in  the  Bayesian  formulation. 
This  usage  is  a  direct  consequence  of  the  theorem,  and  equation  2.3.1  in  particular. 
To  motivate  the  next  theorem,  consider  the  set  defined  via, 


C0  =  (tt  :  p(rr)  <  c(tt)}. 


The  set  Co  is  called  the  natural  continuation  set  for  the  obvious  reason  that  one 
honestly  expects  to  continue  collecting  data  as  long  as  the  risk  is  less  than  the  terminal 
cost.  In  addition  to  Cq,  one  can  define  the  natural  stopping  set  as, 


So  =  {it  :  p(n)  =  e(?r)}, 

without  loss  of  generality  (see  Theorem  2.3.2  below).  It  is  easy  to  see  that  the  two 
sets  are  disjoint  and  that  their  union  is  the  whole  interval,  [0,  l] .  Now  because  the 
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goal  is  to  prove  that  threshold  policies  are  optimal,  it  had  better  be  the  case  that  Cq 
is  an  open  and  connected  set  since  one  believes  intuitively  that  the  stopping  interval 
of  a  threshold  policy  ought  to  be  identical  to  Cq,  if  the  policy  is  Bayesian.  This  and 
more  is  shown  to  be  true  in  the  theorem  below  using  the  simpler  form  of  the  risk 
given  in  2.3.7. 

Theorem  2.3.2  Let  ire  =  prqr^r.  he.,  the  (unique)  point  at  which  e(x)  attains  its 
maximum  value  (e(ire)  =  X+cT)'  Then  there  exist  two  numbers  a*  and  bt  where 

0  <  a*  <  ne  <  b,  <  1,  (2.3.8) 

and  such  that  the  Bayes  risk  satisfies, 

p( tt)  <  e(7r)  Vff  e  (a*, 6*) 

p(x)  =  e(7r)  Vjt  G  (0,  l]  \  (a*,  6*). 

In  addition,  p(ir)  is  continuous  on  [0,  l]. 

Proof: 

In  view  of  the  fact  that  e(7r)  is  concave,  for  any  n  e  [0,  l]  an  easy  consequence  of 
Jensen’s  Inequality  is, 

pM  =  inf  E, r  [  /  c3ds  +  e( 7rT)] 
rer  J  o 

<  inf[^  [  cads  +  e(En[nT})} 

r€T  J  o 

=  inf  En  /  c3ds  +  e(7r) 

•rer  Jo 

=  e(7r)j 

using  the  obvious  fact  that  {7r*  :  t  >  0}  is  a  uniformly  integrable  martingale.  Thus, 

p(7r)  <  e(7r)  W  €  [0,  l].  (2.3.9) 

Now  since  e(0)  =  0  =  e(l),  it  is  clear  from  2.2.6,  the  concavity  of  e(7r),  and  2.3.9  that 
p( 7r)  is  continuous  on  [0,1].  This  inequality  also  justifies  the  definition  of  Sq  given 


above. 
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It  is  now  geometrically  obvious  that  there  exist  the  two  numbers  a*  and  6*  as 
claimed,  nevertheless  one  can  argue  as  follows.  If  So  =  [0, 1],  then  Co  =  0  and 
choosing  a*  =  7r4  =  6*  yields  the  result.  If  So  =  {0,  l},  then  Co  =  (0,  l)  and  choosing 
a*  =  0,  b*  =  1  does  the  same.  Otherwise,  choose  it'  G  So,  0  <  it'  <  1,  and  suppose 
it'  <  7re.  Then  the  following  is  true, 


p(x')  —  e(Tt')  =>•  it  G  Sq  V it  <  it'  <  ite,  (2.3.10) 


for  if  not,  there  exists  it  <  it'  such  that  it  G  Co,  implying  p(ir)  <  e(7r).  But  this 
violates  the  concavity  and  continuity  of  p(it)  on  [0, 1].  To  see  this,  choose  a  G  [0, 1] 
such  that 

Tt  =  a  •  0  +  (1  —  a)ir' 


and  then, 

p(i t)  =  p(a.  ■  0  +  (1  —  a)7r') 

>  ap( 0)  +  (1  -  a)/9(7r') 

=  (1  -  a)e(Tr') 

=  e(7r) 

>  p{*), 

a  contradiction.  The  same  contradiction  is  derived  if  one  supposes  it'  >  ne,  and 
therefore  the  following  is  also  true, 


p{it')  =  e^')  =S>  it  G  So  'i it  >  it'  >  Tte.  (2.3.11) 

Having  established  2.3.10  and  2.3.11,  it  is  now  simple  to  obtain  the  final  case.  It  is 
clear  that  Co  is  an  open  set  so  choose  w',  it"  G  Co,  with  it'  <  it",  and  of  course  assume 
Cq  7^  0.  Now  if  Co  is  not  a  connected  set  then  there  exists  a  G  (0, 1)  such  that 


ait'  -f  (1  -  a) it"  G  So, 
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whence  at  least  n'  €  So  or  tv"  €  So  in  view  of  2.3.10  and  2.3.11.  A  contradiction 
results  in  either  case.  Hence,  Cq  is  an  open  interval  and  therefore  take  a*,  6*  to  be 
its  right-  and  left-hand  endpoints,  respectively.  I 

The  preceding  theorem  has  shown  that  the  natural  continuation  set  is  an  interval 
and  is  therefore  the  primary  candidate  upon  which  to  base  a  threshold  policy,  but 
there  are  special  cases  to  consider.  If  Cq  is  an  empty  interval,  then  p{n)  =  e(jr) 
and  the  identically  zero  stopping  time  is  clearly  Bayesian  optimal.  Obviously  the 
degenerate  threshold  policy  with  thresholds  a  =  ne  —  b  is  also  Bayesian  optimal  in 
this  case.  On  the  other  hand  if  Cq  —  (0, 1)  then  there  is  no  optimal  threshold  policy. 
To  see  this,  pick  an  arbitrary  threshold  policy  u  6  U,  with  thresholds  0  <  a  <  b  <  1. 
Choose  tv  €  dv(a,  6)  and  suppose  ir  ^  {0,  l}.  Then, 

E„[  [  csds  +  e{Kf)\  =  e(7r)  >  p(tt). 

Jo 

Thus  u  is  not  {7r}-optimal  and  therefore  cannot  be  Bayesian  optimal.  Alternatively, 
choose  (a,  b)  —  (0,  l)  to  be  the  thresholds.  Then  f  =  oo,  P„- a.s.  V  tt  G  (a,  b),  and  in 
view  of  (T), 

E„{  c3  ds  +  e(7rf)]  =  oo, 

Jo 

which  is  clearly  not  an  optimal  policy.  Similarly  one  can  show  that  if  one  endpoint 
of  Cq  is  zero  or  one,  then  no  optimal  threshold  policy  exists.  In  the  remainder  of  the 
thesis  therefore,  these  special  cases  are  of  no  consequence  and  it  will  be  assumed  that 
the  class  of  threshold  policies  U  is  further  restricted  to  those  policies  whose  thresholds 
obey  the  following  inequalities, 

(I)  0  <  a  <  7re  <  b  <  1;  a  <  b, 

where  7re  =  argmaxwe[0>i]  e(7f)-  Put  another  way,  under  condition  (I),  if  u  G  U,  then 
0  <  f  <  oo,  P„-a.s.  V 7r  €  (a,  b).  For  the  remainder  of  the  thesis  the  notation  I0 
will  be  used  with  the  implicit  understanding  that  Iq  —  (a,  6)  satisfies  condition  (I). 
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A  short  recapitulation  is  in  order.  Theorem  2.3.1  showed  that  any  policy  could  be 
replaced  by  another  which  would  do  no  worse  and  this  lead  to  a  simpler  expression 
for  the  Bayes  risk.  Using  this  simpler  form  of  the  risk,  the  last  theorem  yielded  more 
insight  into  the  structure  of  the  risk.  But  Theorem  2.3.1  raises  another  point.  In  light 
of  that  theorem,  a  natural  question  to  ask  is  what  policy  results  when  a  threshold 
policy  (f,6)  is  exchanged  for  the  policy  (f,  £),  with  6  as  in  2.3.1.  If  u  is  a  threshold 
policy  based  upon  the  a  posteriori  probability,  the  next  theorem  shows  that  (7,6) 
and  (f ,  6)  are  identical. 

Theorem  2.3.3  Let  Jo  =  (a,  b)  (satisfying  condition  (I)  above),  and  consider  the 
threshold  policy  (f,6)  based  on  Jo  and  {nt  :  t  >  0}  (see  def.  2.1.2).  Then, 

6(u>)  =  6  (to)  Vc o  G  fl, 

with  6  as  in  Theorem  2.3.1. 

Proof: 

Suppose  to  €  fi  is  such  that,  Kf(ui)  (w)  >  6,  and  therefore  6 (to)  =  1.  Then  from 
2.3.1, 

fry  b 

1  —  7Tf  —  1  —  6' 

On  the  other  hand,  6  >  c1/(c°  +  c1),  and  so, 

b  c1 
1  —  b  c°  * 

Thus  JTf  >  b  implies  c°7r?  >  c1(l  —  nr),  and  thus  6(u;)  =  1.  Now  let  w  G  fl  be  such 
that  6(w)  =  1,  implying, 

fry(w)(^)  c}_ 

1  -  fry(u,)M  -  c0’ 

so  that  upon  rearranging  one  obtains, 
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and  therefore  £(w)  =  1.  From  this  one  concludes  that, 

6(o>)  -  1  -4=>-  =  1  Vw  €  0. 

Similarly,  one  can  show, 

6(w)  =  0  •£=$>•  6(w)  =  0  Vo;  G  H. 

This  completes  the  argument.  I 

The  fact  that  threshold  policies  are  invariant  with  respect  to  Theorem  2.3.1 
allows  one  to  prove  the  next  interesting  result.  Clearly,  the  proof  below  depends  on 
this  invariance  property  alone,  but  it  will  be  stated  only  in  terms  of  threshold  policies 
for  emphasis. 

Theorem  2.3.4  Let  uGU  be  a  threshold  policy.  Then, 

P0{6=  1}  +  Pi{£  =  0}  <  1. 


Proof: 

From  Theorem  2.3.3  there  holds, 


En{[  cads  +  w{9,8)\  =  Ew[  [  c3ds-\-e{ *>)], 
Jo  Jo 


and  therefore, 


c°ttPi{8  =  0}  +  c*(l  -  7t)P0{£  =  1}  =  E„[e(irf)]  <  e(7r),  (2.3.12) 

where  the  inequality  follows  as  in  2.3.9.  Suppose  7r  <  7 re  and  thence  e(7r)  =  c°7r.  Then 
2.3.12  implies, 

0  <  c°7r(l  -Pi{8  =  0})  -  cx(  1  -  tt)P0{^  =  1} 

<  c°7re(l  -Pi{6  =  0})  -  c^l  -  ne)P0{6  =  1} 

=  «(*,)[! -(P„{*=1}  + Pi {*  =  <>})]. 
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The  same  inequality  is  obtained  if  one  supposes  that  tt  >  7re.  Thus  except  in  the 
excluded  cases  (c°c1  =  0)  it  follows  from  the  fact  that  e(7re)  >  0  that 

Po{6  =  1}  +  P^S  =  0}  <  1, 

and  the  theorem  is  shown,  | 

Corollary  Suppose  u  =  (r,  6)  G  U  is  Bayesian  and 

Po{S  =  1}  +  Pi{6  =  0}  =  1. 

Then,  the  degenerate  threshold  policy  with  thresholds  a  =  b  =  ire  is  Bayesian  with 
risk  p(tt)  =  e(7r). 

Proof: 

Observe  that, 

E*{  [  c*  ds  +  c°tcPi{6  =  0}  +  c*(l  —  7t)P0{6  =  l}]  =  p(tt)  <  e(7r)  V7re[0,  l], 

Jo 

Thus, 

0  <  Ev  [  cs  ds  <  e(7re)[l  -  (P0{6  =  1}  +  Pi{6  =  0})], 

Jo 

arguing  as  above;  and  therefore, 

E,r  I  ca  ds  =  0. 

Jo 

As  a  result, 

p{ tt)  =  c°nPi{6  =  0}  +  c^l  -  7t)P0{£  =  1}, 
so  if  7T  >  ne  one  can  show 


p(?r)  >  cx(l  -  7r)  =  e(7r); 

and  similarly  if  it  <  ne, 


p(ir)  >  c°7r  =  e(7r). 
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Hence  p(n)  =  e[ir)  V7T  G  [0, 1].  This  same  risk  is  achieved  with  the  stated  degenerate 
threshold  policy.  This  completes  the  proof.  I 

Obviously,  it  is  necessary  that  any  Bayesian  optimal  policy  satisfy  the  last  two 
theorems.  In  the  general  excessive  function  theory  approach  [SHIRYAYEV]  to 
optimal  stopping,  one  defines  at  the  outset  the  Bayesian  stopping  time  candidate  in 
terms  of  the  natural  stopping  set  as, 

t  =  inf{t  >  0  :  7Tt  G  50} 

with  the  intention  of  using  Theorem  2.3.1  to  define  the  decision  component.  Such  a 
policy  is  seen  to  satisfy  the  consequents  of  the  last  two  theorems.  Assuming  one  can 
prove  under  given  conditions  that  t  is  Bayesian,  then  the  task  necessary  to  demon¬ 
strate  that  a  threshold  policy  is  optimal  becomes  precisely  one  of  showing  that  Co 
is  open  and  connected.  These  properties  of  Co  have  already  been  established  and 
so  r  as  defined  above  is  indeed  the  stopping  time  of  a  threshold  policy.  Now,  to 
prove  directly  that  this  stopping  time  is  Bayesian,  one  can  use  the  excessive  function 
approach,  but  this  requires  ‘rather  deep  results  from  the  general  theory  of  Markov 
processes  and  martingales’  \loc.  cit.  113],  in  general.  Alternatively,  one  might  try  to 
discover  simple  sufficient  conditions  which  yield  optimality  and  then  show  them  to 
hold  in  a  particular  application.  In  the  next  chapter,  such  a  set  of  conditions  is  set 
forth,  one  which  is  sufficient  to  prove  the  existence  of  a  Bayesian  threshold  policy. 
This  concludes  the  section  and  chapter. 


Chapter  III 

Sufficient  Conditions  for  Bayesian  Optimality 


3.0  Introduction 

The  purpose  of  this  chapter  is  to  identify  the  conditions  under  which  threshold  policies 
based  on  the  a  posteriori  probability  are  Bayesian  optimal,  and  indicate  how  to  use 
them  in  a  particular  application  to  demonstrate  optimality  as  well  as  compute  the 
thresholds.  The  chapter  is  organized  as  follows. 

Section  3.1  gives  a  set  of  three  conditions  which  are  sufficient  to  guarantee  that 
threshold  policies  are  Bayesian.  Two  lemmas  are  given  which  help  characterize  the 
risk  and  the  continuation  interval.  The  main  theorem,  which  is  based  on  these  lemmas, 
then  demonstrates  the  sufficiency  of  the  conditions  mentioned. 

In  a  particular  application,  the  first  two  sufficient  conditions  are  obtained  by  con¬ 
struction  as  will  be  shown  in  Chapter  IV.  A  direct  verification  of  the  third  condition 
is  difficult  in  general,  so  in  section  3.2,  auxiliary  conditions  are  given  which  allow  one 
to  obtain  the  third  condition  from  the  other  two.  The  chapter  ends  with  a  section 
discussing  an  alternative  set  of  sufficient  conditions. 

3.1  Sufficient  Conditions 

The  purpose  of  this  section  is  to  spell  out  a  set  of  three  conditions  which  are  sufficient 
to  prove  that  there  exists  a  unique  threshold  policy  which  is  Bayesian.  From  the 
last  chapter,  it  is  clear  that  the  process  of  fundamental  interest  is  the  a  posteriori 
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probability  process  {irt  :  t  >  0}.  In  the  remainder  of  this  chapter  it  will  be  assumed 
that  {irt  :  t  >  0}  is  in  fact  a  standard  Markov  process  [DYNKIN]. 

Since  the  goal  of  this  thesis  is  to  demonstrate  the  optimality  of  threshold  policies, 
it  is  natural  that  the  topic  of  first  exit  times  of  a  process  from  an  open  interval  should 
arise.  In  turn  then,  it  might  be  expected  that  the  notion  of  characteristic  operator 
should  appear.  This  is  indeed  the  case. 

Let  U  denote  the  characteristic  operator  [DYNKIN]  for  the  a  posteriori  prob- 

t 

ability  process  {7rf  :  t  >  0}  and  D{U)  denote  its  domain  of  definition.  Now  given 
that  {irt  :  t  >  0}  is  as  above,  suppose  that  r  :  [0,  l]  — ►  IR.  satisfies  the  hypotheses  of 
a  generalized  Ito  rule  for  this  particular  (n't  :  t  >  0},  so  that  one  can  write 

r(7rr)  -  r(7r0)  =  [  Ur(ira)ds  +  MT,  (3.1.1) 

Jo 

\ 

for  all  (P„,  Jt)-stopping  times  satisfying  E„  [r]  <  oo,  where  M  is  some  Pw-local  martin¬ 
gale.  Finally,  let  J0  C  [0, 1],  strictly,  be  some  nonempty,  open  interval  and  r  £  V{JJ) 
be  such  a  function.  Consider  the  following  set  of  conditions  on  the  pair  (r,  70): 


(Cl): 

Ur{ir)  =  — c(  tt) 

Vtt€  (0, 1); 

(02): 

r(rr)  =  e(7r) 

Vtt  £  dnlo] 

(C3): 

r(7r)  <  e(?r) 

Vtt  ^  d^Io. 

In  a  particular  application,  the  first  two  conditions  pose  a  generalized  Stefan 
problem  [SHIRYAYEV]  which  does  not  have  a  unique  solution  in  general.  The  third 
condition  serves  to  constrain  the  problem  so  that  its  solution  is  essentially  unique,  as 
will  be  described  below.  Notice  that  the  function  r(7r)  is  like  a  Dynkin  function  in 
that  it  satisfies  (Cl),  is  similar  to  the  risk  function  in  as  much  as  it  satisfies  (Cl)  and 
(C2),  but  differs  from  both  in  the  additional  requirement  (C3).  The  remainder  of  this 
section  is  devoted  to  proving  that  the  conditions  (Cl)-(C3)  are  sufficient  to  guarantee 
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that  there  exists  a  threshold  policy  which  is  Bayesian.  The  first  lemma  goes  a  long 
way  towards  showing  this. 

Lemma  3.1.1  Suppose  there  exists  a  pair  (r,  70)  satisfying  the  conditions  (C1)-(C3). 
Then, 

p(ir)  =  r(ir)  Vtt  G  [Jo]*,  (3.1.2) 

where  p(-)  is  the  Bayes  risk. 

Proof: 

First  note  that  (C2)  and  (C3)  together  imply, 


e(ir)  >  r(ff)  Vff  6  [0, 1]. 


(3.1.3) 


Next,  in  view  of  (Cl),  one  obtains, 

P(”)  =  mf  p(ir,r) 

=  mf  EW[J  c(ir3)  ds  +  e(nr)\  (3.1.4) 

=  r(jr)  +  inf  E„[e(irr)  -  r(?rT)]  Vtt  G  [0, 1]. 

t€T 

Hence,  combining  3.1.4  and  3.1.5  yields, 


p{n)  >  r( n)  V7r  G  [0,  l].  (3.1.5) 

On  the  other  hand,  if  one  defines  the  (Pn,  0t)-stopping  time  f  as, 

f  =  inf{t  >  0  :  7rt  £  Iq},  (3.1.6) 

then,  from  (Cl)  and  (C2)  it  follows  that, 

p(ir,  f)  =  r(jr)  Vtt  G  [70]w.  (3.1.7) 

Thus,  3.1.5  and  3.1.6  together  yield, 


p(ir)  =  r(ir) 


Vtt  G  [Jq]*-, 


(3.1.8) 
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and  the  lemma  is  shown.  | 

Because  of  3.1.8  and  (C3),  it  is  convenient  and  suggestive  to  call  r(-)  the  subrisk. 
Likewise,  it  is  appropriate  to  call  J0  the  continuation  interval. 

The  next  lemma  shows  that  if  there  are  two  pairs,  say  (r,  J0)  and  ( s ,  Jo),  which 
satisfy  the  conditions  (C1-C3),  then  Iq  =  J0  and  r(7r)  =  s(tt),  V7 r  E  [Jo]*.  This  is 
called  essential  uniqueness  because,  as  it  will  be  shown,  the  nonuniqueness  of  r(7r) 
off  of  [Jo]*  is  irrelevant  to  any  questions  concerning  the  risk.  This  is  fortunate  since 
r(i r)  is  not  in  general  unique  there. 

Lemma  3.1.2  Suppose  there  exists  a  pair  (r,  J0)  such  that  the  conditions  (C1-C3) 
hold.  Then  ( r,  J0 )  is  essentially  unique  and  in  particular,  J0  =  Co. 

Proof: 

To  prove  the  theorem  it  is  only  necessary  to  show  that  J0  =  Co,  in  view  of  the 
previous  lemma.  Pick  any  7 r  E  Jo,  and  observe  that 

p(tt)  =  r(7r)  <  e(7r), 

in  view  of  Lemma  3.1.1  and  (C3),  and  so  n  E  C0.  Thus  J0  C  C0,  and  claim  J0  =  C0, 
for  if  not  dl0  C  C0.  In  this  case,  choose  7r  E  dlo  and  note  tt  G  d*J0  C  [Jo]*.  From 
Lemma  3.1.1  and  (C2)  then 

p(  7r)  =  r(  7r)  =  e(7r). 

But  n  G  C0  also,  implying  p(7r)  <  e(7r)  which  yields  the  contradiction.  I 

Corollary  Under  the  same  assumptions, 

p(ir)  =  e(n)  Vtt  ^  J0  (3.1.9) 

Proof:  Immmediate  from  the  lemma  and  Theorem  2.3.2.  I 

The  preceding  lemmas  are  the  main  ingredients  to  the  principal  optimality  result 
proved  next.  To  summarize,  Lemma  3.1.1  showed  that  the  three  conditions  imply  that 
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the  risk  and  the  subrisk  are  identical  on  the  7r-closure  of  the  continuation  interval. 
Hence,  by  itself  the  lemma  suggests  that  threshold  policies  are  at  least  [Jo]*. -Bayesian. 
Lemma  3.1.2  and  its  corollary  showed  that  the  risk  equals  the  terminal  cost  outside 
of  the  continuation  interval.  Putting  the  two  lemmas  together  yields  the  following. 

Theorem  3.1.3  Suppose  there  exists  a  pair  (r,  Jo)  satisfying  the  conditions  (C1-C3). 
Then  the  threshold  policy  based  on  Iq  is  Bayesian. 

Proof: 

Suppose  (Cl),(C2),  and  (C3)  hold,  and  define  r*  =  f  as  in  3.1.6.  Then  3.1.7  and 
3.1.8  give, 

p(ir,T*)  =  p(n)  We  [Jo]*-,  (3.1.10) 


i.e.,  r*  solves  the  optimal  stopping  problem  for  all  tt  e  [Jo]*.  Since  r*  is  of  threshold 
type,  it  follows  that  the  policy  u*  =  ( r*,6 *)  exists  and  is  [J0]*-Bayesian,  where  S*  is 
given  by, 


f  1  if  7 rT.  >  b 
\  0  if  7Tt*  <  a, 


for  Jo  =  (a,  b). 

It  follows  from  Lemma  3.1.2  that, 


p(n)  =  e(ir)  W  ^  Jo.  (3.1.11) 

On  the  other  hand,  r*  =  0P*-a.s.W  ^  Jo,  and  then, 

p(ir,r*)  =  E„[J  c(ns)ds  +  e(7rT.)] 

=  ^[O-he(Tr)]  (3-1-12) 

=  «(*)• 

Hence  t*  solves  the  optimal  stopping  problem  for  all  7r  ^  Jo.  Combining  this  fact 
with  the  above  argument  for  ir  G  [Jo]*-  shows  that  there  exists  a  policy,  in  particular 
u*  =  ( t*,S *),  t*  as  in  3.1.6  and  6*  as  above,  such  that  u*  is  Bayesian.  I 
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Thus,  in  order  to  prove  that  there  exists  a  Bayesian  threshold  policy,  one  need 
only  exhibit  a  subrisk-interval  pair  satisfying  the  three  conditions.  In  practice,  one 
constructs'a  family  of  pairs  whose  members  all  satisfy  the  first  two  conditions.  The 
remaining  difficulty  is  to  show  that  there  exists  within  this  family  at  least  one  pair 
which  enjoys  the  third  condition.  An  existence  proof  is  all  that  is  necessary  here  in 
view  of  the  essential  uniqueness  property.  In  the  next  section,  auxiliary  conditions 
are  given  which  allow  one  to  obtain  (C3)  from  (Cl),  (C2)  and  properties  of  the  pair 
which  are  more  readily  verifiable. 
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3.2  Auxiliary  Conditions 

In  the  last  section,  the  goal  was  to  demonstrate  the  sufficiency  of  the  conditions  (Cl- 
C3)  in  proving  that  threshold  policies  are  Bayesian.  The  emphasis  in  this  section  is 
to  replace  condition  (C3)  with  properties  of  the  subrisk  and  the  continuation  interval 
which  are  more  easily  verified.  Such  auxiliary  conditions  play  a  primary  role  in  ap¬ 
plications,  by  providing  characterizations  which  guide  the  search  for  the  (essentially) 
unique  subrisk  and  continuation  interval  pair  obeying  (C1-C3). 

The  next  theorem  is  very  important  in  applications  in  that  it  provides  a  way  to 
demonstrate  the  concavity  of  the  subrisk  based  on  its  smoothness  properties,  quite  a 

t 

bargain  since  a  direct  proof  of  concavity  can  be  much  more  difficult  than  a  verification 

of  smoothness.  Note  then  the  intimate  connection  between  this  theorem  and  the 

condition  (C3)  in  an  applications  setting. 

\ 

Theorem  3.2.1  Let  r(i r)  exist  satisfying  (Cl),  and  suppose  r(7r)  is  continuous  on 
[/<>]».  Then  r( 7r)  is  concave  on  [Jo]*  if  r  E  C2+([/0]w). 

Proof: 

The  proof  will  proceed  by  contradiction.  Suppose  r(7r)  is  not  concave  on  [Jo]*. 
Then  it  is  strictly  convex  on  some  subinterval  J  C  [Jo]*  in  view  of  the  fact  that 
r  €  (C°nC2+)([J0]*).  Define, 

tj  = inf{£  >  0  :  7 rt  ^  J},  (3.2.1) 

and  suppose  7To  €  J,  P*- a.s.  Now,  consider  any  stopping  time  a  E  T ,  where  0  <  cr  < 
tj,  P*- a.s.  Since  r(7r)  is  strictly  convex  on  J,  one  obtains, 

Ev[r{iTa)}  >  r(E^[na})  =r( tt)  n  E  J,  (3.2.2) 

where  the  inequality  is  an  application  of  Jensen’s  Inequality,  and  the  equality  follows 
since  {7rt  :  t  >  0}  is  a  uniformly  integrable  (P*,  Ot)-martingale.  On  the  other  hand, 


Ur(n)  <  0 


E„[r(TrT)\  <  t(tt)  Vr  E  T,  tt  E  [Jo],, 


(3.2.3) 
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i.e.,  condition  (Cl)  guarantees  that  Ev[r[ ttt)|  is  bounded  above  by  its  initial  value 
(indeed,  it  can  be  shown  that  r( nt)  is  a  (PT,  Ot)-supermartingale).  As  a  result,  the 
strict  convexity  of  r  is  a  clear  contradiction,  and  therefore  r(7r)  is  concave  on  [Iq]*..  1 

Corollary  If  in  addition  r(7r)  satisfies  ( C2 ),  then  r( 7r)  is  continuous  on  [70]ff 
Proof:  Immediate  from  the  theorem.  I 


The  next  theorem  is  used  in  the  applications  to  follow  to  obtain  the  condition 
(C3)  from  the  conditions  (Cl)  and  (C2),  and  properties  of  the  subrisk  function. 


Theorem  3.2.2  Let  there  exist  a  pair  (r,  Iq)  satisfying  the  conditions  (Cl)  and  ( C2 ), 
and  suppose  r  £  (C°  D  C2+) ([/o]T)  is  right  continuously  differentiable  on  (0, 1).  With 
I0  =  (a,  b)  assume  that, 

r'[a )  <  e'(a) 


r'(b  )  >  e'(b). 


(3.2.4) 


Then, 


r( 7r)  <  e(n)  'i  it  £  Iq. 


Proof: 

From  Theorem  3.2.1,  r(7r)  is  concave  and  continuous  on  Thus, 


where, 


Likewise, 


where, 


r(ir)  <  la(ir)  V7re[/0]T 

la[n)  =  r(a )  +  r'(a)(ir  -  a). 

r{ir)<lb{n)  V7re[/o]^ 

lb{ir)  =  r(b)  +  r'(b~)(ir  -  a). 


From  condition  (C2), 


la(n)  —  e(a )  +r'(a)(n  -  a), 
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and  so  from  3.2.4  it  follows  that 


r(w)  <  e(n)  VttE  (a,  7rc] . 

Similarly, 

r(ic)  <  e[it)  V7re[7re,a), 

and  so  the  subrisk  is  upper  bounded  by  the  terminal  cost  on  [Jo]*-.  Now  arguing  as 
in  Theorem  2.3.2  with  the  subrisk  instead  of  the  risk,  one  can  show  that  there  exist 
two  numbers  aT,  bT  such  that, 

0  <  a  <  aP  <  7Te  <  br  <  b  <  1; 

where, 

r(7r)  <  e(7r),  V7rE(ar,6r); 

r(7r)  =  e(7r),  V7r  e  (a,  ar]  U  [br,  b). 

Now  if  one  can  show  that  ar  —  a  and  br  =  6,  then  the  proof  will  be  complete.  To  this 
end  suppose  br  <  b  and  choose  7r  G  [ br,b )  C  Jo-  Then 

U{r{ir)  —  e(7r))  =  0, 


and  so, 


Ue( tt)  <  0, 


in  view  of  condition  (Cl).  But  e(7r)  is  affine  on  [6r,  1]  yielding  the  contradiction, 


Ue(n)  =  0, 


since  the  characteristic  operator  of  a  martingale  clearly  annihilates  affine  functions. 
The  same  contradiction  is  derived  if  ar  >  a.  I 
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3.3  Discussion 

This  chapter  has  set  down  and  investigated  a  set  of  conditions  sufficient  to  demon¬ 
strate  Bayesian  optimality.  It  should  be  clear  that  these  conditions  are  not  the  most 
simple.  Indeed,  the  simplest  (nontrivial)  sufficient  conditions  are  given  as  follows. 
Suppose  that  for  any  r  £  T  one  can  find  a  threshold  stopping  time  f  £  T  satisfying, 


e{nT)  >  e(n>); 


ca  ds  >  En 


Cg  ds, 


(3.3.1) 

(3.3.2) 


for  all  7T  £  [0, 1].  Then  clearly  one  need  only  consider  threshold  policies,  for  if  a 
Bayesian  policy  exists,  then  one  can  by  assumption  find  a  threshold  policy  which  is 
Bayesian  optimal  (see  2.3.7).  This  is  the  essence  of  the  argument  used  in  [LaVIGNA] 
to  exhibit  necessary  and  sufficient  conditions  for  the  existence  of  an  (a,  6)-Bayesian 
threshold  policy  when  the  observations  arise  from  a  diffusion  observed  through  an¬ 
other  diffusion  and  for  a  particular  choice  of  the  cost  {ct  :  t  >  0}.  The  condition  3.3.1 
is  obtained  therein  with  equality  by  choosing  the  thresholds  so  as  to  match  the  error 
probabilities  of  the  threshold  policy  to  any  arbitrary  (admissible)  rival  policy.f  The 
matching  is  made  possible  by  the  fact  that  the  error  probabilities  are  related  to  the 
thresholds  by  continuous,  invertible  mappings.  In  fact,  these  mappings  are  simple 
and  explicit  (see  equation  17.124  in  [LIPTSER  &;  SHIRYAYEY  78].  They  are  the 
well  known  Wald  approximations  [WALD]  which  are  exact  in  the  diffusion  case  where 
there  is  no  overshoot. 


The  condition  3.3.2  is  shown  to  be  in  effect  as  follows.  First,  one  computes  a 
lower  bound  for  the  left  hand  side  of  3.3.2  for  all  t  £  T  using  Jensen’s  Inequality 
(see  [WALD],  [LIPTSER  &;  SHIRYAYEY  78]).  In  fact,  this  lower  bound  holds  ir- 


f  Be  aware  that  the  hypothesis  test  there  is  based  on  the  likelihood  ratio.  However, 
the  likelihood  ratio  is  related  to  the  a  posteriori  probability  through  a  bijection  and 
thus  the  tests  are  equivalent,  only  the  thresholds  differ.  See  Appendix  I. 
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respective  of  the  continuity  of  either  the  state  space  of  the  {nt  :  t  >  0}  process  or 
the  time  parameter  set  (compare  with  Theorem  2.3.1),  for  a  proper  choice  of  run¬ 
ning  cost.  However,'  in  the  diffusion  case  where  continuity  holds  for  both,  one  can 
show  that  the  right  hand  side  of  3.3.2  equals  the  lower  bound  (see  equation  17.115 
[LIPTSER  &  SHIRYAYEV]).  Therefore,  the  two  conditions  hold  for  diffusions  in 

general,  with  the  usual  cost  caveat.  Thus  the  sufficient  conditions  3.3. 1-3. 3. 2  can 

\ 

\ 

be  shown  to  hold  true  in  the  diffusion  case  and  therefore  one  need  only  search  for 
an  optimal  policy  from  among  those  o^  threshold  type.  The  necessary  and  sufficient 
conditions  for  the  existence  of  a  (a,  5)-Bayesian  threshold  policy  are  then  obtained 
by  minimizing  the  risk  directly  over  the  thresholds.  Given  that  the  threshold  set  is 
open,  if  a  minimum  exists  it  can  be  obtained  by  setting  the  derivatives  of  the  thresh¬ 
old  risk  with  respect  to  the  two  thresholds  equal  to  zero.  This  leads  to  a  system  of 
transcendental  equations  for  which  the  existence  and  uniqueness  of  a  fixed  point  is 
given  as  a  proposition  in  the  proof  of  Theorem  5  in  [SHIRYAYEV  p.185]  because  the 
same  system  is  obtained  via  the  so-called  smooth  pasting  conditions  arising  in  the 
excessive  function  theoretic  approach-! 

Now  unfortunately,  the  sufficient  conditions  3.3. 1-3. 3. 2  are  useless  in  all  but  the 
diffusion  case  for  two  reasons.  First,  matching  the  error  probabilities  by  choosing 
the  thresholds  can  be  done  explicitly  in  only  this  case,  and  in  virtually  all  others 
the  best  that  one  can  hope  for  is  a  proof  of  the  existence  of  such  thresholds.  The 
matching  process  is  simple  for  diffusions  because  of  their  sample  path  continuity  and 
it  is  the  lack  of  this  continuity  which  causes  great  analytical  difficulty  for  jump- type 
processes.  Second,  the  lower  bound  mentioned  above  is  attained  in  only  the  diffusion 
case.  Again,  this  is  intrinsic  to  diffusions  because  of  the  continuity  of  their  sample 

|  Incidentally,  the  proposition  given  in  [SHIRYAYEV]  is  correct  but  its  proof  is  not 
(see  equations  4.77-4.78  there  and  the  subsequent  comments).  Consult  Theorem  4.1.1 
in  the  next  chapter  for  a  proof  of  an  equivalent  proposition. 
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paths.  In  essence,  the  Wald-Jensen  bound,  if  you  will,  is  simply  not  tight  enough  for 
processes  with  noncontinuous  sample  paths. 

Given  this  state  of  affairs,  the  attempt  was  made  to  find  an  alternative  set  of 
sufficient  conditions  which  might  yield  optimality  for  both  diffusions  and  processes 
with  noncontinuous  sample  paths  (but  continuous  time  parameter).  The  alternative 
set  of  conditions  were  given  in  section  3.1  and  our  attention  now  turns  to  using  them 
in  applications. 

This  concludes  the  chapter.  In  the  next,  the  results  of  the  previous  chapters  will 
be  used  to  solve  four  problems  arising  in  applications.  The  first  two  deal  with  the 
Bayesian  problem  when  the  observations  come  from  a  diffusion,  and  the  second  two 
when  one  is  observing  a  point  process. 


Chapter  IV 

Applications  in  Sequential  Testing 


4.0  Introduction 

In  this  chapter  four  problems  in  Bayesian  sequential  hypothesis  testing  will  be  formu¬ 
lated  and  solved.  Given  the  framework  established  in  the  second  chapter,  formulation 
here  means  specifying  the  two  hypothesized  probability  measures,  and  the  observation 
cr-algebra.  In  section  4.1  two  problems  are  considered  where  one  observes  a  diffusion 
and  seeks  to  minimize  a  given  Bayes  risk.  In  the  first,  it  is  assumed  that  one  of  two 
constant  drifts  is  responsible  for  the  observations.  Actually,  the  detection  version  of 
the  hypothesis  testing  problem  is  solved  because  it  is  mathematically  equivalent  while 
notationally  simpler.  A  constant,  positive  cost-rate  is  chosen  for  this  problem  which 
is  proportional  to  the  the  square  of  the  nonzero  drift.  It  is  shown  that  a  threshold 
policy  based  on  the  a  posteriori  probability  is  optimal,  and  the  risk  and  thresholds 
are  given.  The  second  problem  considered  is  the  case  where  a  diffusion  with  a  drift 
which  is  assumed  to  be  merely  progressively  measurable  is  the  basis  for  the  detec¬ 
tion  problem.  It  is  shown  that,  if  one  chooses  a  constant,  positive  cost-rate  which  is 
proportional  to  the  square  of  the  stochastic  drift,  then  again,  a  threshold  policy  is 
optimal.  In  fact,  it  is  shown  that  the  risk  and  thresholds  are  identical  to  those  in  the 
homogeneous  case. 

In  section  4.2,  two  additional  formulations  are  given.  The  first  considers  the  case 
where  under  each  hypothesis,  it  is  assumed  that  the  observations  are  due  to  a  constant 


50 


§4.0  Introduction 


51 


rate  Poisson  process.  Here  the  hypothesis  testing  problem  is  presented  because  the 
detection  problem — a  constant  rate  versus  a  rate  of  one — is  no  real  notational  bargain. 
A  threshold  policy  is  shown  to  be  Bayesian,  and  explicit  formulas  for  the  risk  and 
thresholds  are  computed.  The  last  problem  deals  with  the  situation  where  a  point 
process  with  one  of  two  predictable  rates  is  assumed  responsible  for  the  observations. 
It  is  shown  that  if  one  chooses  a  constant  positive  'c^ost-rate  which  is  proportional  to 
the  difference  of  the  stochastic  rates,  then  there  exists  a  Bayesian  threshold  policy. 
Again,  the  risk  and  the  thresholds  are  the  same  as  those  in  the  homogeneous  case. 
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4.1  Diffusion  Observations 

This  section  will  draw  on  the  formulation  and  notation  laid  down  in  sections  2.1 
and  2.2  on  the  Bayesian  formulation  to  the  sequential  hypothesis  testing  problem. 
Following  the  briefest  review,  the  problem  will  be  specialized  to  the  case  where  the 
filtration  history  arises  from  observations  of  a  continous  semimartingale  driven  by  a 
standard  Wiener  process.  ' 

Let  7 r  €  [0, 1]  be  arbitrary  throughout  the  remainder  of  the  section,  unless  other¬ 
wise  noted.  This  will  save  the  continual  trouble  of  having  to  restate  the  obvious,  and 
this  will  not  be  done,  except  for  emphasis  in  some  of  the  numbered  equations. 

Recall  that  on  the  measurable  space,  (0,  7)  there  is  given  a  family  of  probability 
measures  {Pv  :  0  <  ir  <  l},  satisfying  2.2.2,  and  a  {0o,0i}-valued  random  variable, 
9,  with  the  a  priori  probability  distribution  given  in  2.2.1.  Based  on  the  observation 

I 

filtration,  0t,  one  wants  to  choose  a  value  for  9  which  minimizes  the  risk,  defined  in 
2.2.4.  Values  for  9  are  chosen  by  a  two-step  procedure.  First,  the  decision  to  terminate 
the  observation  procedure  is  made  according  to  a  (Pn,  0t)-stopping  time,  say  r,  and 
then  the  value  for  9  is  chosen  according  to  a  (P*,  0T) -binary  random  variable,  say 
6  (see  definition  2.1.1).  A  particular  application  begins  by  specifying  the  nature  of 
{0t  •  t  >  0},  the  <T-algebra  generated  by  the  observations. 

It  is  assumed  that  the  random  variable  9  is  unobservable,  but  that  one  can  observe 
an  initially  zero  continuous  random  process  {yt  :  t  >  0}  whose  statistics  under  each 
of  the  hypotheses — 9  =  9q,  9  —  9\ — are  governed  by  the  probability  measures  P0  and 
Pi,  respectively.  To  wit,  for  each  w  G  fl,  the  observation  process  has  the  stochastic 
differentials, 

{h°(cj)dt  +  dwt(cj),  if  9(u>)  =  90; 
h\(<jj)dt  +  dwt(u),  if  0{w)  =  0\, 

where,  {wt  :  t  >  0}  is  a  (P*,  /t)-Standard  Wiener  process  which  is  P* -independent 
of  9,  and  where  {h\  :  t  >  0}  is  an  7t- progressive  process  satisfying,  P,[/o  <  00 > 
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t  =  0, 1.  Now  the  above  hypothesis  testing  problem  can  be  easily  shown,  without  loss 
of  generality,  to  be  equivalent  to  a  detection  problem,  and  so,  letting  0Q  =  0,  and 
6\  =  1,  the  problem  can  be  recast  in  the  detection  format,  in  which  case  {yt  :  t  >  0} 
has  the  (Pv,  ?t) -stochastic  differential 


dyt  =  Ohtdt  +  dwt  t  >  0, 


(4.1.1) 


with  the  same  requirements  on  {ht  :  t  >  0},  and  {w t  :  t  >  0}  just  mentioned. 

From  4.1.1,  one  takes  the  observation  filtration,  0t,  to  be  the  er-algebra  generated 
by  the  {yt  :  t  >  0}  process.  Having  done  this,  recall  from  section  2.2  that  the  a 
posteriori  probability  is  7T*  =  P„{0  =  l}0t},  t  >  0.  This  permits  the  total  average 
risk  to  be  given  as 

=  E„{[  c(ir3)  ds  +  e(7rT)],  (4.1.2) 

Jo 

where  ct(7r)  is  a  given  cost-rate  function,  and  e(?r)  is  the  terminal  cost  as  in  2.3.3. 
Note  that  4.1.2  is  free  of  6,  as  achieved  in  section  2.3. 

Having  defined  the  observation  filtration,  the  immediate  goal  is  to  compute  a 
stochastic  differential  for  the  { nt  :  t  >  0}  process.  From  4.1.1  it  is  clear  that  P0  <C  Pi, 
so  take, 

(4.1.3) 

to  be  the  likelihood  ratio  for  the  problem,  i.e.,  it  is  the  Radon-Nikodym  derivative  of 
the  0t-restriction  of  the  measure  Pi  with  respect  to  the  Ot-restriction  of  the  measure 
Pq.  If  7r  =  1,  then  Pv{irt  =  1}  =  1  with  P„  probability  one,  for  all  t  >  0.  Suppose 
7r  <  1;  in  Appendix  I,  it  is  shown  in  this  case  that  (see  A.1.11), 


TTt  = 


-  <£(At), 


and  so,  applying  the  Ito  differentiation  rule  one  obtains, 

7T* (1  —  7Ta)  rt 


Kt 


(4.1.4) 


(4.1.5) 
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The  likelihood  ratio  for  this  problem  is  well  known  and  is  given  by, 


At  =  exp{  J  hadya  J  h2ds}, 


where, 


ht  =  Ei[ht\Ot]' 


From  4.1.3  and  4.1.6,  one  can  obtain, 

At  =  1  +  /  haAadya. 
Jo 

Now,  if  one  observes  that  (see  A.1.7), 


(4.1.6) 


(4.1.7) 


(4.1.8) 


-^ir[^At|0t]  —  TVfhf, 


(4.1.9) 


then  a  straightforward  calculation  shows  that  there  exists  a  ( Pn ,  Ot)-standard  Wiener 
martingale,  say  {iZ>t  :  t  >  0},  such  that  {yt  :  t  >  0}  has  stochastic  differential, 

dyt  =  7 tthtdt  +  dwt.  (4.1.10) 


Combining  4.1.5,  4.1.8,  and  4.1.10  yields, 

TTt  —  tt0  =  /  7ra(l  -  na)hadya  -  /  it2a{\  -  tv S)h2ads 
Jo  Jo 

—  /  7T,(l  —  7 va)hadwa  Vt  >  0, 

Jo 

and  so, 

dtvt  =  7Ti(l  -  7r a)h3dwa. 


(4.1.11) 


(4.1.12) 


It  is  clear  from  4.1.12  that  since  {wt  :  t  >  0}  is  a  continuous  martingale  that 
d„Io  =  dlo  =  {a,  6},  if  Iq  =  (ai&)  is  some  interval  such  that  a  <  tvq  <  b,  P^- a.s.,  and 
0  <  a  <  b  <  1.  It  may  not  be  clear  that  {7rt  :  t  >  0}  can  be  guaranteed  to  satisfy 
Theorem  1.1.1,  i.e.,  can  be  guaranteed  to  escape  any  such  interval.  To  see  that  this 
is  so,  first  note  that  7Tt  =  $(£*),  where  $  :  1R  ->  [0, 1]  is  the  bijective  function  given 
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implicitly  by  4.1.4  and  the  relation,  Lt  =  log  i.e.,  $(L)  =  ^(log(L));  L  is  called 
the  log-likelihood  ratio.  Now,  equations  4.1.6  and  4.1.10  show, 

dLt  =  (vt  -  \)h\dt  +  htwt  t>  0,  (4.1.13) 

which  yields  the  (Pff,  Ot)-compensator  for  L  as, 

At  =  (*t  -  t>  0.  '(4.1.14) 

As  prerequisites  to  Theorem  1.1.1,  it  is  assumed  that, 

(Al):  PwlM&l  =  oo}  =  1  V7re[0, 1]; 

(A2):  Pv{\At  \  =  oo}  =  0  Vtt  €  [0,  l],  Mt  <  oo. 

These  assumptions  are  necessary  to  ensure  that  {7rt  :  t  >  0}  will  escape  Iq  in  fi¬ 
nite  time  P*- a.s.,  and  thence  guarantee  that  the  threshold  policy  with  continuation 
interval  Iq  will  eventually  terminate.  Since  P»  P*-,  *  =  0, 1,  it  follows  that, 

Pr{\A^\  =  oo}  =  1  Pi{\A^\  =  oo}  =  1  *  =  0,1.  (4.1.15) 

In  view  of  4.1.15,  assumptions  equivalent  to  (Al)  and  (A2)  can  be  recast  in  terms  of 
the  measures  Pi,  *  =  0, 1  as, 

(Al)'  :  Pi{  f  h]ds  =  oo}  =  1  *  =  0,1; 

Jo 

(A2)'  :  Pi{  f  h\ds  =  oo}  =  0  *  =  0,1,  Vt  <  oo. 

Jo 

The  third  condition  of  Theorem  1.1.1  can  be  shown  directly  as, 

|E.[Af]|  =  |(l-*)E„[^]+rf,[^l| 

=  l(*  -  »)®>l£  \ hldsl  +  nEtiJ' 

=  \(*  -^Eo^  +  nE^W 

^lE^Afl-wEMfl  +  nEMiW 


(4.1.18) 


Chapter  IV :  Applications  in  Sequential  Testing 


56 


Hence,  by  Theorem  1.1.1,  L  is  guaranteed  to  escape  any  open  interval  in  finite  time 
P^-a.s.,  and  since  nt  =  $(£t)  where  $  is  bijective,  then  the  same  holds  true  for  the 
a  posteriori  probability. 

Next,  let  r  :  1R.  — ►  [0,  l]  be  some  twice  continuously  differentiable  function.  An 
application  of  the  Ito  rule  gives, 

rt  i 

7T2(l  -  ir3)2h]r"(TTe)ds  +  /  7ra(l  -  Tra)hsr' (ir3)dw3,  (4.1.19) 

Jo 

using  4.1.12. 

Suppose  ht  =  h,  t  >  0,  where  h  is  some  deterministic,  nonzero  constant.  Thus, 
(Al)/  and  (A2)*  hold.  In  addition,  suppose  that  the  cost-rate  function  is  given  as 
c(tt)  =  ch 2,  with  c  a  positive  deterministic  constant;  this  choice  of  cost-rate  entails 
no  loss  of  generality  since  c  >  0  is  otherwise  arbitrary.  In  addition,  it  follows  that 
condition  (T)  holds,  thus  £?w[/0°°  csds\  =  oo.  This  is  a  classic  Bayesian  set-up:  a  coii- 
stant  running  cost  risk  with  a  homogeneous  diffusion.  From  4.1.19,  the  characteristic 
operator  of  the  a  posteriori  probability  process  is  found  to  be, 

Ur(n)  =  h27r2(l  —  7r)2r" (7r)  W  G  (0, 1).  (4.1.20) 


'(» rt)  ~  r{ir0)  =  [ 
Jo 


The  application  of  the  results  in  Chapter  III  begins  by  setting  up  the  problem  specified 
by  conditions  (Cl)  and  (C2),  which  is: 

'  Ur(iv)  =  — ch 2  Vtt  €  (0,  l); 

< 

r(7r)  =  e(?r)  w  G  {a,  b}. 

Substituting  4.1.20  into  the  above  and  dividing  by  h 2  yields, 

(  7T2(1  —  7r)2r"(7r)  =  — c  Vtt  G  (0, 1); 

(4.1.21) 


(  r(ir)  =  e(r)  n  e  {a,b}, 

which  is  a  family  of  ODE’s  indexed  by  the  boundary  points  {a,  6},  a  <  b.  Motivated 
by  Theorem  3.2.1,  it  is  required  that  a  solution  to  4.1.21  for  o,  b  given  at  least  satisfies, 


r(7r)  is  continuous  W  G  [a,  b}. 


(4.1.22) 
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Clearly,  the  solution  to  4.1.21-22  exists,  is  unique,  and  is  easily  obtained  in  closed 
form  as, 

,(*)  =  ^e(o)  +  j-— — [e(6)  -  d(b)  +  <*(<.)]  +  d(n)  -  d(a),  (4.1.23) 

o  —  a  o  a 

where, 

d{ 7r)  =  c(l  -  2tt)  log  -  — — ■ 

Now  it  is  clear  that  r(?r)  is  concave  for  all  tt  G  (0,  l)  directly  from  4.1.21.  However, 
it  is  also  true  that  4.1.21  shows  that  r  G  C2+{E),  with  E  =  [a,  6],  and  so  invoking 
Theorem  3.2.1,  it  follows  that  r(fi-)  is  concave  for  all  ir  G  [a,  b].  Obviously,  this  weaker 
concavity  argument  is  trivial  and  unnecessary,  but  it  is  given  for  pedagogical  reasons— 
to  show  at  which  step  Theorem  3.2.1  is  applied.  It  will  turn  out  that  the  concavity  of 
the  subrisk  is  not  a  ‘free  gift’  in  the  discontinuous  martingale  case  considered  in  the 
next  section. 

Since  r(7r)  is  concave  for  all  w  G  (0,1),  then  condition  (C3)  is  satisfied  if  one 
imposes, 

r'(a)  =  e'(a) ,  r'(6)  =  «'(&).  (4.1.24) 

It  is  obvious  that  4.1.24  guarantees  (C3),  nevertheless,  an  appeal  to  Theorem  3.2.2 
can  be  made.  The  solution  {a*,b*}  to  4.1.24  is  easily  shown  to  exist  (see  below). 
Its  uniqueness  is  also  easily  shown,  but  follows  a  fortiori  in  view  of  Lemma  3.1.2 
simply  from  its  existence.  Again,  in  the  jump  process  case,  this  is  a  welcome  device 
to  proving  uniqueness  because  the  existence  proof  will  be  difficult  enough. 

In  summary,  the  three  sufficient  conditions,  (C1-C3),  have  been  shown  to  be  in 
force.  Thus,  the  following  theorem  has  been  demonstrated  to  be  true. 

Theorem  4.2.1  In  the  problem  of  sequential  detection  based  on  observations  of  the 
homogeneous  diffusion  process, 


yt  --  8ht  +  wt 


hi-  0;  t  >  0, 
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with  running  cost, 


E„[f  cads\  —  Ev[ch2r\,  (4.1.25) 

Jo 

there  exists  a  Bayesian  rule,  u*  =  ( r*,S *),  which  is  a  threshold  policy  given  by, 

t*  =  inf{i  >  0  :  irt  £  (a*, 6*)} 

c*  f  1,  if  IT T.  >  b*; 

\  0,  if  7rT*  <  a*. 


(4.1.26) 


The  Bayes  risk  is  given  by, 


it  (£  (a*,b*); 
7r  E  (a* ,b*), 


with  subrisk, 


*/\  b*  —  7r  .  7 r  —  a 

r  (7r)  =  - - «(a*)  +  - 

v  ‘  b*  -  a*  K  '  1 


a*  L 


e(&*)  -  d(6*)  +  d(a*)  +  d{ ic)  -  d{a*), 


and. 


d(ir)  =  c(l  —  2?r)  log  — - — . 

1  —  7T 

(4.1.27) 

The  constants  a*  and  b*  are  uniquely  determined  from  the  system  of  transcendental 

equations, 

e'(a*)-d'(a*)  =  f(a*,b*y, 

e'(b*)-d'(b*)  =  f(a*,b*), 

(4.1.28) 

with, 

jfa  o  _  e(b )  “  e(«)  -  ( d(b )  ~  «*(®)) 

b  —  a 

(4.1.29) 

where  a* ,  b* 

satisfy  condition  (I),  i.e.. 

0  <  a*  <  ?re  <  b*  <  1;  a*<b*. 


Proof: 

Given  largely  in  the  preceding  discussion.  Note  that  the  conditions  (T),  (Al),  and 
(A2)  are  satisfied  trivially  because  h  0  and  because  c(7r)  =  ch2,  with  0  <  c  <  oo. 
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Also  note  that  r*  <  oo,  P*  —  a.s.,  V7T  E  [0, 1],  in  view  of  Theorem  1.1.1.  The  system 
4.1.28  to  solve  for  the  optimal  thresholds  a*,  b*,  is  obtained  from  4.1.23  and  4.1.24. 
In  order  to  show  that  a*,  b*  exist  satisfying  4.1.28,  begin  by  transforming  the  system 
to  the  following  form: 


c1  —  d'(b)  =  c°  —  d'(a) ; 
c1  +  bd'[b )  —  d[b )  =  ad' (a)  —  d(a) . 


(4.1.30) 


The  origin  of  the  first  equality  should  be  clear;  obtain  the  second  by  eliminating 
c°  from  the  system  4.1.28.  To  simplify  the  remaining  computations  let  A  =  y£~, 
B  —  j~i,  and  thus  the  requirement  0  <  a  <  b  <  1  becomes  0  <  A  <  B  <  oo.  Upon 
substituting  these  values  into  4.1.30  and  rearranging  there  holds, f 

C°  +  c1  +  c[~  -  B  -  2log£]  =  c[~  -  A  —  2  log  A] ; 

B  A  (4.1.31) 

c1  —  c\B  +  log  B\  =  —  c[A  +  log  A] . 

Next,  eliminate  c 1  in  the  first  equation,  divide  both  sides  of  both  the  result  and  the 
second  equation  by  c  >  0,  and  rearrange  to  get, 


.o  1  ,  1  1,1 

e  +fl+log5  =  A  +  logI; 


c1  +  A  +  log  A  =  B  +  log  B  , 


(4.1.32) 


where  c°  =  c°/c  and  c1  =  c1/c.  If  one  considers  4.1.32  as  an  implicit  definition  of  the 
mappings  A  (->  Bo(A)  and  A  i-»  Bi(A),  respectively,  it  follows  that  Bq(A )  and  B\{A) 
have  the  same  terminal  behavior  as  A  j  0  and  as  A  |  oo.  Such  behavior  is  undesirable 
since  our  goal  is  the  equivalent  of  finding  a  fixed  point  to  Bq(A)  =  Bi(A).  With  this 
in  mind,  rearrange  4.1.32  as, 


,  B 

+logi 

,  B 

+  !og- 


(4.1.33) 


f  Compare  4.1.31  with  equations  4.77  and  4.78  in  [SHIRYAYEV]. 
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Define  R  =  4  and  make  the  substitution, 


R  -  1  ,  „  .0 

— - h  log  R  =  c°  ; 

BR"r  1  +  log  R  =  c1 , 


(4.1.34) 


with  the  requirements  0<B<oo,  l<iE<oo.  Consider  the  first  equation  in 
4.1.34  and  the  implicit  mapping  R  i-v  Bq(R).  One  can  show  via  the  Implicit  Function 

t 

Theorem  that  Bq(R)  G  C1(1,oo)  where  -Bo(l)  —  0  and, 


On  the  other  hand,  it  also  follows  that  Bi(R)  G  C'1(l,oo)  where  £?i(l)  =  oo  and, 

B'^  =  wh(1+sP)<0- 


Consequently,  there  exists  a  unique  R  satisfying 


0  <  -Bo(i2)  =  Bi(R)  <  oo  ;  1  <  R  <  oo.  (4.1.35) 


Chaining  back  through  the  argument,  it  has  been  shown  that  there  is  a  unique  so¬ 
lution  (a*, b*)  satisfying  4.1.28  and  also  0  <  a*  <  b*  <  1.  From  Lemma  3.1.2  and 
Lemma  2.3.2,  it  follows  in  addition  that  condition  (I)  is  satisfied.  This  completes  the 
proof.  I 

To  facilitate  the  numerical  calculation  of  the  thresholds,  begin  by  subtracting 
the  first  equation  in  4.1.34  from  the  second  to  obtain, 


B 


R  —  1 
R 


R-  1 
B 


=  c1 


and  thence, 


yielding, 


(4.1.36) 
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Replacing  B  by  4.1.36  in  the  second  equation  of  4.1.34  and  defining  S  =  \/R  >  1, 


one  can  obtain  the  following  compact  expression, 


(5  -  i)  ^cp  [sinh-1 


4-  2  log  S  =  c1 . 


(4.1.37) 


If  one  computes  a  solution  to  4.1.37,  then  the  optimal  thresholds  (a*,  b*)  are  obtained 
from, 

1  c^-c0 


a  =  s exp 

B  —  S  exp 
*  A 


sinh 


-l 


2  S  —  S~1 


Lh-> 

V25-5-VJ 


(4.1.38) 


a 


b*  = 


1  +  A'  . 
B 

1  +  B  ' 


In  the  special  case  where  c°  =  c1  >  0,  it  follows  that  B  =  S  =  1/A  and  B  satisfies, 


J3  —  —  +  2  log  B  —  c  —  c  . 
B 


(4.1.39) 


Note  in  this  case  that  b*  =  1  —  a*,  and  0  <  a*  <  |  <  b*  <  1.  As  an  example,  if 
c°  =  c1  where, 

c°  =  (3-  i +  2log(3))  •  c, 


then, 


In  general,  4.1.39  can  be  used  to  obtain  an  initial  guess  for  an  iterative  method 
involving  4.1.37,  or  can  be  used  as  an  approximation  when  c°  «  c1. 

Notice  that  the  drift  coefficient  has  been  effectively  ‘factored  out’  of  the  problem 
as  a  result  of  choosing  the  cost-rate  as  in  4.1.25.  A  particular  choice  of  the  cost-rate 
constant  for  an  instance  of  the  above  problem  is  purely  a  design  decision,  modulo  the 
positivity  requirement.  There  is  no  choice  which  is  uniformly  good  for  all  problems. 
However,  the  choice  made  herein,  c(7r)  =  ch 2,  is  intuitively  satisfying  since  it  not 
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only  enjoys  positivity,  but  moreover  exhibits  the  appealing  property  of  penalizing  the 
detector  more,  or  less,  depending  upon  whether  the  magnitude  of  the  drift  is  greater, 
or  smaller  than  one,  and  does  so  according  to  the  square  of  the  magnitude.  One  might 
say  that  this  choice  of  cost-rate  is  reasonable  in  that  it  reflects  a  designer’s  modest 
desire  to  expect  better  performance  from  the  detector  (faster  decisions  on  average)  in 
‘favorable’  problems  (large  drifts  to  detect),  and  to  allow  worse  performance  (slower 
decisions  on  average)  in  ‘hard’  problems  (drifts  close  to  zero).  Apparently,  such  a 
designer  believes  “not  all  filtrations  are  created  equal”,  to  put  it  colloquially. 

As  mentioned,  a  by-product  of  this  choice  of  cost  is  that  the  drift  is  factored  out 
of  the  risk,  and  therefore  the  risk  is  independent  of  any  particular  drift.  This  is  a 
significant  advantage  because  it  implies  immediately  that  the  results  of  Theorem  4.1.1 
can  be  extended  to  a  more  general  case.  To  see  this,  first  consider  that  the  proper 
generalization  of  the  cost  choice  discussed  above  is  to  take  the  running  cost  as, 


£*[  f  cada]  =  En[f  ch]ds ], 
Jo  Jo 


(4.1.40) 


in  the  case  where  the  drift  of  {yt  :  t  >  0}  is  in  general  only  known  to  be  (P„,  7t)- 

A 

progressive.  For  this  choice  of  cost,  it  is  clear  that  ct  =  ch f,  c  >  0,  t  >  0,  is  a 
(P*,  0t)- progressive  process.  Having  made  this  choice  of  running  cost,  one  can  prove 
the  following. 


Theorem  4.1.2  Assume  (Al)  and  (A2)  hold.  In  the  problem  of  sequential  detection 
based  on  observations  of  the  process  (see  4.1.1), 

yt  =  9  f  h3ds  +  Wt  t>  0,  (4.1.41) 

Jo 


with  running  cost, 


c  >  0,  t  >  0, 


(4.1.42) 


the  threshold  policy,  u*  =  (r*,S*),  given  in  the  homogeneous  case,  is  Bayesian.  More¬ 
over,  the  risk  and  thresholds  are  exactly  the  same. 
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Proof: 

First  note  that  (T)  follows  from  (Al). 
The  computation  of  the  risk  is  given  by, 


pM  =  inf  E„[  cads  +  e{%T)\ 

tGT  J0 

=  inf  E„[  ch2a  ds  +  e(irT)\ 
reT  Jq 

=  inf  Ev[-  [  h23Ur*(na)ds  +  e(irT)}, 

JQ 

with  r*  as  Theorem  4.1.1  and  where  U  is  the  operator  as  in  4.1.21.  Hence, 


(4.1.43) 


p(ir)  =  r*(7r)  +  inf  E„[e(irT)  -  r*  (*>)]  =  r*  (tt),  (4.1.44) 

t€ET 

but  this  is  the  same  risk  as  in  the  homogeneous  case,  and  so  the  result  is  shown.  ■ 
A  special  case  of  the  above  result  is  given  in  [LaVIGNA],  using  an  argument 
based  upon  a  theorem  due  to  Shiryayev  [LIPTSER  h  SHIRYAYEV  77]  for  the  gener¬ 
alized  problem  of  Wald  sequential  detection  [WALD].  The  theorem  above  was  proven 
independently  and  in  fact  can  be  used  to  obtain  the  generalized  Wald  result  sim¬ 
ply  and  directly,  using  an  argument  which  is  formally  equivalent  to  the  method  of 
LeCam  used  in  the  discrete-time  case  [LEHMANN].  The  proof  given  here  has  an¬ 
other  advantage  in  that  it  makes  it  clear  how  to  generalize  the  Bayesian  problem 
to  include  other  cost-rates  and  thence  also  to  extend  the  Wald  results.  As  a  final 
note,  the  reader  should  be  aware  that  the  formulas  given  here  are  different  from  those 
appearing  elsewhere,  but  lead  to  the  same  results. 

This  concludes  the  section.  In  the  next,  the  Bayesian  problem  is  considered  for 
the  homogeneous  Poisson  case,  which  has  interesting  similarities  with  the  homoge¬ 
neous  diffusion  case.  Lastly,  a  more  general  theorem  is  obtained  from  the  homoge¬ 
neous  result  by  choosing  the  proper  cost,  in  much  the  same  way  that  the  homogeneous 
result  was  extended  in  this  section. 
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4.2  Point  Process  Observations 

This  section  will  consider  the  sequential  hypothesis  testing  problem  from  the 
Bayesian  viewpoint  when  the  observations  arise  from  a  continuous-time  point  process. 
The  notation  and  basic  set-up  are  given  in  sections  2.1  and  2.2. 

Recall  that  on  a  measurable  space  (fi,T),  there  is  given  a  family  of  probability 
measures  (Pn  :  0  <  tt  <  1},  satisfying  2.2.2,  and  a  binary  valued  random  variable, 
0,  with  the  a  priori  probability  distribution  given  in  2.2.1.  Based  on  the  observation 
filtration,  Ot,  one  wants  to  choose  a  value  for  0  which  minimizes  the  risk,  defined  in 
2.2.4.  The  manner  in  which  values  for  0  are  chosen  has  two  parts.  First,  the  decision  to 
terminate  the  observation  procedure  is  made  according  to  a  [Pv,  0t)-stopping  time, 
say  r,  and  second,  a  value  for  0  is  chosen  according  to  a  (Pn,  0T)-binary  random 
variable,  say  8  (see  definition  2.1.1).  A  particular  application  begins  by  specifying 
the  nature  of  {0t  :  t  >  0},  the  <r-algebra  generated  by  the  observations. 

It  is  assumed  that  the  random  variable  0  is  unobservable,  but  that  one  can  observe 
a  counting  process  {n*  :  t  >  0}  whose  statistics  under  each  of  the  hypotheses — 0  —  0, 
0  =  1 — are  governed  by  the  probability  measures  Po  and  Pi,  respectively.  Specifically, 
for  each  w  €  0,  the  observation  process  is  an  initially  zero  counting  process  with 
semimartingale  representation, 

So  if  0(u)  =  0; 

nt(w)  = 

„  fo  Aj(w)ds  +  ?ntM,  if  0(u>)  =  1, 

where,  {m*  :  t  >  0}  is  a  (Pw,  /t)-martingale  which  is  Pw-independent  of  0,  and  where 
{AJ  :  t  >  0}  is  an  Tt- predictable  process  satisfying  Ei  J0*  |A*  |ds  <  oo,  *  =  0, 1.  Thus, 
it  is  assumed  that  the  observation  process  is  a  (Pw,  Tt) -semimartingale  with  stochastic 
differential, 

dnt  =  (0Aj  +  (1  -  0)\°t)dt  +  dmt  t>  0,  (4.2.1) 

while  the  observation  filtration  is  the  o-algebra  generated  by  this  process. 
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The  first  step  is  to  compute  a  ( Pn ,  Ot)-semimartingale  representation  for  the  a 
posteriori  probability,  irt  =  Pv{6  —  l|Ot},  1  >  0,  as  a  precursor  to  minimizing  the 
total  average  risk  given  in  2.3.7.  To  begin,  note  that  the  likelihood  ratio  for  the 
problem  is  well  known  and  is  given  by, 

At  =  exp [  f  log  \^dn3~  f  (Aj  -  A°)ds]  t  >  0,  (4.2.2) 

Jo  A9  Jo 


where, 


At  =  Ei[X\\0t-}  t  >  0. 


(4.2.3) 


From  4.2.2  it  follows  that, 


A  A  _ 

dAt  =  At-  At  ~-At  ( dnt  -  A °dt) . 
At 


Now,  using  4.1.4,  one  obtains, 


7Tf  ~  K0 


_  f*  M1  ~  *«) 


dAcs+  A^(Aa), 


0<*<t 


and  from  4.2.4 


dAct  = -A3(X]  -  \°3)ds. 


(4.2.4) 


(4.2.5) 


(4.2.6) 


The  computation  for  A<f>(At)  is  given  as, 


A*(At)  = 


(1+  I^A0(1  + 


/ 1  ^  AAt 

=  7Tf_(l  -  7TtJ  — - 

At- 

A  -  A  _ 

_  7Tt-(l  ~  7Tt) 


(4.2.7) 


where  the  last  line  also  serves  to  define  (ut  :  t  >  0}.  In  addition,  note  that, 


(1  -  7 Tt-)ut 

Ut  +  7Tt_ 


(4.2.8) 
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Combining  4.2.5,  4.2.6,  4.2.7,  and  4.2.8  yields, 


7Tt  —  7T0  =  —  f  (A*  —  A°)7Ts(1  —  ir3)ds  +  ^  ~ 

(4.2.9) 

=  -  [  [\1a-\°a)na[l-na)ds+  f 
Jo  JO 

where  the  second  line  follows  since  {mt  :  t  >  0},  and  therefore  {nt  :  t  >  0},  has 
unity  jumps.  Now,  if  one  observes  that  (see  A.1.7), 


E„[0\\  +  (1  -  =  X]irt-  +  A°(l  -  7rt_),  (4.2.10) 

then  a  straightforward  calculation  shows  that  there  exists  a  (Pw,  Ot) -martingale,  say 
{mt  :  t  >  0},  such  that  {nt  :  t  >  0}  has  the  (P, r,  Ot)-stochastic  differential, 


dnt  =  (A J TTt—  +  A°(l  -  irt-))dt  +  dmt, 


(4.2.11) 


Substituting  4.2.11  into  4.2.9  gives, 

fz  7r,_(l  -  tts 

Tct-n0=  /  - ; - 

J0  ua  +  7Va — 


*  «•-(!—  *.-) 


(4.2.12) 


Having  computed  4.2.12,  the  next  step  is  to  investigate  under  what  conditions  a 
process  as  in  4.2.12  can  be  guaranteed  to  escape  an  interval  and  hence  serve  as  the 
basis  of  a  threshold  policy. 

Consider  an  interval  7o  =  (a,  b )  such  that  0  <  c  <  b  <  1,  and  a  <  no  <  b,  P„- a.s. 
From  4.2.2,  4.2.11,  and  the  relation  Lt  =  log  Af,  there  holds, 


dLt  =  [(Aj7Tt_  +  A?(l  -  7rt_))  log  -  (Aj  -  A?)]dt  +  log  dmt , 

At  At 

yielding  the  (P„,  Ot)-compensator  for  L  as, 


(4.2.13) 


A?  =  (Aj^t-  +  A° (1  -  *_))  log  |  -  (A l  -  A?). 


(4.2.14) 


§4.2  Point  Process  Observations 


67 


As  in  section  4.1,  the  assumptions  (Al)  and  (A2)  must  hold  in  order  to  apply  Theo¬ 
rem  1.1.1  and  thus  ensure  that  L  will  escape  Jo  in  finite  time  Pv- a.s.  In  view  of  4.1.5, 
the  assumptions  (Al)  and  (A2)  can  be  recast  in  terms  of  the  measures  P{,  i  =  0, 1, 
equivalently  as, 

n  oo 

(Al)"  :  Pi{  /  A l)ds  =  oo}  =  1  i  =  0, 1; 

Jo 


(A2)"  :  P{{  j  ^(A*  l,  A \)ds  =  oo}  =  0  i  =  0, 1,  Vf  <  oo. 

where, 

^(A1,  A0)  =  A1  log  ^  -  (A1  -  A0).  (4.2.15) 

Notice,  for  A0,  A1  >  0, 


p(A1,A°)  =  A1log^-(AI-A°) 

>  A1  log  +  A1  log  =  0, 


(4.2.16) 


and  similarly, 

^(A0,A1)  =  -A°log^-(A°-A1) 

>  -(A1  -  A0)  +  (A1  -  A0)  =  0. 


(4.2.17) 


The  third  condition  of  Theorem  1.1.1  is  an  easy  consequence  of  4.2.16  and  4.2.17  using 
the  same  steps  as  in  4.1.18.  Thus  by  Theorem  1.1.1,  L  is  guaranteed  to  escape  Jo  in 
finite  time  P„- a.s.,  and  since  nt  =  $(Lt),  where  $  is  bijective,  then  the  same  holds 
true  for  the  a  posteriori  probability.  A  difficulty  however,  is  that  the  7r-boundary 
of  Jo  is  not  simply  its  endpoints  as  in  the  diffusion  case,  and  worse,  it  is  stochastic 
unless  Ut  is  deterministic  for  all  t  >  0.  Therefore,  although  conditions  have  been  given 
under  which  {? r*  :  t  >  0}  can  be  guaranteed  to  escape  intervals,  the  7r-boundary  of 
the  process  is  difficult  to  work  with  in  the  general  case. 

Avoiding  this  difficulty  for  the  moment,  it  is  desired  to  compute  the  characteristic 
operator  for  (7Tt  :  t  >  0}.  So,  assuming  r  e  C1+(E),  with  E  =  (0,  l),  an  application 


f 

Jo 
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of  Theorem  1.1.2  gives, 

r(nt)  -  r(ir0)  - 

~  ^ ° )  [  -tts(1  -  rra)r'(ira)  +  (u,  +  y*)[f  (~^~7r«)  ~  r(^)l]ds 

+  /  [  rf  U-*~  ~t 1  -irs_)  -  r{TTa-)\dfht, 

Jo  Vtt,-  +  7r,-  > 

(4.2.18) 

using  4.2.11  and  4.2.12. 

Now  consider  the  case  where  A*  =  A1,  A°  =  A0,  t  >  0,  are  deterministic  con¬ 
stants.  Assume  that  these  constants  satisfy  A1  >  A0  >  0,  without  loss  of  generality. 
Thus  (Al)"  and  (A2)”  hold.  Also  suppose  that  the  cost-rate  function  is  given  as 
c(tt)  =  c(Ax  —  A0),  a  positive  deterministic  constant;  thus  E„[f^°  c(7rs)ds]  =  oo  and 
therefore  condition  (T)  holds.  The  terminal  cost  is  e(7r),  as  usual. 

In  view  of  4.2.9  with 

A0 

it  is  clear  that  d~ Iq  =  {o},  i.e.,  {7rt  :  t  >  0}  will  exit  I0  continuously  on  the  right. 
However,  since  {7rt  :  t  >  0}  will  exit  Iq  on  the  left  only  by  jumping,  then  it  follows 
that  3+ 10  =  [6,  ^±|6).  To  see  this,  consider  that  whenever  -irt  E  [b  —  b),  it  has 

the  potential  of  getting  into  3+io  because  its  jump  size  is  at  that  time. 

Letting  denote  the  new  state  arrived  at  by  jumping  from  7rt_,  i.e., 

,  *■*-(!  —  *■*-)  U+1  _ 

IjTtt-  —  7Tt_  -I - ; -  —  — ; - Kt~, 

U  +  7 Tt_  U  +  7 rt_ 


yields, 


3*10  =  {a,  [6,  Eft)}, 


as  the  7r-boundary  of  /o,  for  Io  as  defined  above. 

From  4.2.18,  the  characteristic  operator  of  {7rt  :  t  >  0}  is  found  to  be, 


f/r(7r)  =  (A1 -A0) [-7r(l-7r)r'(7r)-|-(u+7r)  r(U~]~  *7r)-r(7r)  ]  V7r  €  (0, 1).  (4.2.19) 

U  i  7r 
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The  application  of  the  results  in  sections  3.1  and  3.2  begins  by  setting  up  the  problem 
specified  by  conditions  (Cl)  and  (C2),  which  is: 

Ur( tt)  =  — c(A*  —  A0)  Vtt  E  (0, 1); 

< 

r{ir)  =  e(ir)  it  E  {a,  [6,  £6)}, 

choosing  a  particular  form  for  the  cost  rate  constant  without  loss  of  generality  (see 
diffusion  case  in  section  4.1).  Substituting  4.2.19  into  the  above  and  dividing  by 
(A1  —  A0)  >  0  yields 

— tt(1  —  7r)r'(7r)  +  (u  +  7r)  [r(— -tt)  —  r(?r)]  =  — c  W  E  (0, 1) 

L  “  +  7r  J  (4.2.20) 

r( 7r)  =  e(7r)  tt  E  {a,  [b,  £6)}, 

which  is  a  family  of  functional  advance  differential  equations,  indexed  by  the  boundary 
points  {a,  b}.  Motivated  by  Theorem  3.2.1,  it  is  required  that  a  solution  to  4.2.20 
be  continuous  for  all  tt  E  [a,  £6).  Letting  U  represent  the  operator  in  the  equivalent 
problem,  it  was  shown  in  Theorem  1.1.2  that  for  every  0  <  a  <  b  <  1,  that  there 
exists  a  unique  solution  to, 

UR{ tt)  =  -c  Vtt  G  (0,6) 

R(n)  =  e(7r)  V7r  G  {a,  [6,  S6)} 

with  R(tt)  continuous  for  all  7T  G  (0,6).  Also,  in  the  theorem  to  follow  it  is  shown 
that  there  exists  a  right-continuous  function,  d( 7r),  satisfying, 

U  d(ir)  =  — c  V7rG(0, 1) 

d(7r)=0  Vttg[6,E6) 

d( 7r)  <0  V7r  >  E6, 

for  any  6  G  (0,  l).  So,  take  D(n)  =  d(i r)  +  e(n),  tt  G  [6,  l),  and  construct, 

r(Ti-)  =  R(i r)  l{^-<6>  +  D{ it)  1{W>6}. 

This  gives  a  family  of  right-continuous  functions,  3J  =  {  r(- ;  a,  6)  0  <  a  <  6  <  1} 

satisfying, 

U  r(n)  =  —  c  V7tg(0,  6) 


r(7r)  =  e(7r)  V7r  G  {a,  [6,  E6)}, 
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which  are  strictly  continuous  for  all  7r  <  b,  and  all  of  which  furnish  the  conditions 
(Cl)  and  (C2).  In  addition,  by  the  corollary  to  Theorem  1.2.1,  every  r  G  3?  also 
enjoys,  r  G  C2+(E),  with  E  =  (0,6),  and  as  a  result,  by  the  corollary  to  Theorem 
3.2.2  it  follows  that  every  r  G  SR  is  concave  on  E.  Recall  also  from  the  corollary  to 
Theorem  1.2.1  that,  r'(ir)  is  continuous  for  all  7r  G  (0,6),  with  the  possible  exception 
of  jt  =  E-16.  This  implies, 

r'(ir~)  >  r'(T)  n  —  £-16,  Vr  G  SR, 

since  every  r  G  SR  is  concave.  Solving  for  the  derivative  in  4.2.20, 

c  +  (u  +  7r)[r(^7r)  -r(7r)] 
r  w  = - ^(1^) - ’ 

and  evaluating  it  at  B±,  with  B  =  £-16,  yields, 

,.n±.  _  c+(.  +  B)|r(t*)-r(B)] 

B(1  —  B) 

with  the  immediate  consequence, 

r(6-)  >  r(6),  Vr  G  SR  (4.2.24) 

in  view  of  (4.2.21).  Now  if  any  r  G  SR  satisfies  4.2.24  with  strict  inequality,  then 
obviously  such  a  member  violates  condition  (C3).  As  a  result,  one  is  only  interested 
in  the  subfamily, 

SRi  =  {r  G  SR  :  r(6“;  a, 6)  =  e(6)  }.  (4.2.25) 

Again,  the  corollary  to  Theorem  1.2.1  has  something  to  say.  If  r  G  SR1;  then  r(7r)  is 
twice  continuously  differentiable  at  it  =  a  (unless  6  =  ^^o)  and  therefore,  in  order 
to  satisfy  condition  (C3),  it  is  necessary  that  r( n)  satisfy, 


(4.2.22) 


(4.2.23) 


(4.2.21) 


r'(o)  =  e'(a)  r  G  SRi. 


(4.2.26) 
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Thus,  if  there  exists  r  G  3f2  =  {  r  G  3?i  :  r'(a;a, b)  =  e'(a)},  then  r  satisfies  the 
conditions  (Cl),  (C2),  and  (C3),  and  in  addition,  is  concave  on  [(a, 6)]ff.  In  this  case, 
r(ir)  is  the  subrisk  being  sought. 

An  interesting  subtlety  has  arisen  here  however.  Contrasted  with  the  subrisk 
problem  in  the  diffusion  case,  it  appears  that  the  problem  here  is  ‘overdetermined’. 
Consider  the  diffusion  case.  There,  the  two  free  constants  of  the  second  order  dif¬ 
ferential  equation  were  chosen  to  satisfy  the  boundary  condition  (C2).  Then,  the 
boundary  points  ‘a’  and  ‘6’  were  chosen  as  the  solution  to  a  system  of  two  equations 
in  two  unknowns  whose  unique  solution  was  guaranteed  to  match  the  derivatives  at 
the  boundary,  and  thence  yield  condition  (C3).  Here,  the  single  free  constant  of  the 
first  order  differential  equation  is  chosen  to  satisfy  the  boundary  condition  at  7T  =  a, 
and  this  leaves  only  ‘a’  and  ‘6’  with  which  to  satisfy  the  boundary  condition  at  7 r  =  6 
and  the  condition  on  both  derivatives  to  obtain  condition  (C3).  The  pathway  out  of 
this  apparent  difficulty  is  provided  by  the  following  lemma. 


Lemma  4.2.1  Let  r  G  5J.  Then  r'(b  )  >  e'(b). 
Proof: 

From  4.2.22, 


-in  - 


u  +  b  .u  + 1  .  u  +  6  ... 

e(rrrb)-  ur—ixr(b  ) 


6(1-6)  6(1-6)  Ku  +  b  '  6(1-6) 

ue(b)  u  +  b 


6(1  -  6)  6(1  -  6) 


in 


=  c'm  +  jjjiljK*)  -  r(6-)]. 

Since  r  G  Sft,  it  is  clear  from  the  last  line  and  4.2.24,  that 


r'(b~)  > 


6(1  -  6) 


+  e'(6)>e'(6) 


(4.2.27) 


(4.2.28) 


since  c  >  0  is  assumed. 


I 
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Corollary:If  r  E  3?i,  then, 


r’(b~) 


c 


+  e\b). 


(4.2.29) 


Proof:  Immediate  from  4.2.27. 


Hence,  if  there  exists  r  E  3?25  then  it  indeed  satisfies  the  conditions  (Cl-3).  In 
summary  then,  the  problem  to  solve  in  the  search  for  the  subrisk  is  to  prove  the 
existence  of  any  pair  a*,  6*  satisfying, 


K&*  =  «(M; 

r'(a.;a,,&,)  =  e'(a*), 


(4.2.30) 


or, 

r'(67 i  «.,*,)  =  -t- 

r'(a*;a„,6„)  =  e'(a*), 

where  r  E  3?.  Recall  that  the  uniqueness  of  the  pair  will  follow  from  its  existence  in 
view  of  Theorem  3.1.4.  Be  aware  that  4.2.30  is  a  well-posed  problem  because  it  has 
been  shown  that  3?  is  not  empty,  i.e.,  for  any  0  <  a  <  b  <  1,  there  exists  a  solution 
to  4.2.20.  Proving  that  a*,  6*  exist  satisfying  4.2.30  is  somewhat  involved,  and  so 
is  given  in  Appendix  n.  Based  upon  their  existence,  the  following  theorem  can  be 
shown. 


Theorem  4.2.1  In  the  problem  of  sequential  hypothesis  testing,  based  on  observa¬ 
tions  of  the  homogeneous  Poisson  process, 


n t  =  (0A1  4-  (l  —  0)A°)  t  +  mt  A*  >  A^1,  t  >  0,. 


K[T  c(ns)  dS]  =  ^[c(A1-A°)r], 
Jo 


with  running  cost, 
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there  exists  a  Bayesian  rule,  u*  =  (t*,£*)>  which  is  a  threshold  policy  given  by, 


r„  =  inf{  t  >  0  :  7Tt  ^  (a*, 6*)  } 


The  Bayes  risk  is  given  by, 

pOO  =  | 


if  7Tt  >  6* 

if  7rT  <  a*. 


e(7r) 

r*(7r)  7re(a*,&*), 


and  the  subrisk,  r*( 7r)  =  r(7r;  a*,  6*)  is  given  by, 


j  a*, 6*)  tt  <  6»; 

7T>6„ 


with, 

S(7r;b)  =  e(7r)  +  d(7r;b); 

d(n;  b)  =  C{ A1  (1  -  *r)  +  A°jr)  (1  +  JVj(ir)), 
where  JV6(7r)  is  the  integer  such  that  x(b )  -  x(ir)  -  1  <  iVfc(7r)  <  5(6)  -  and, 

A1  -  A0 


*00  =  log[j 


7T 


X  = 


7T 


l°g 


;  C  =  c 


A°A1 


The  definition  of  the  subrisk,  is  completed  by  specifying, 

R( 7r;  a,  b)  =  d(7r;  6)  +  D(ir,  b)  +  K(a,  b)  H(tt,  6); 

D{jr,  b)  =  A1  (1  —  7r)  D0(tt,  h)  +  A°7r  Z?i(tt;  6); 

#(*■;  6)  =  A1  (1  -  it)  H0(ir,  b)  +  A°tt  Hi  (t r;  b), 
with  Hi,  Di,  i  =  0, 1  defined  by, 

Hi{ir,b)  =  e~,'ii^)  ]T  *-“/-[(*(&)- *00  “  n)*'<e  Vi]n -t 

711 


n=0 


N6(ir)-1 


Di(ir,b)  =  — C'e1/<(5(i)-5(ir)~1)  £  e"1"*  I] 

where, 


(-1 )’ 


n=0 


m= 0 


mi 


■[(5(6)  -  x(7r)  -  n  -  , 


A*  log  |tr  . 
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The  constant  K  (a,  b )  is  defined  via, 

e(a)  -  [d{a\  b)  +  D(a\  6)] 

(  ’  J  E(a;b ) 

The  pair  (a*,&*)  is  the  unique  solution  to 

r'(b~;a*,b+)  =  e'(6»)  4-  fjfz  &  ); 
r'(a.;a*,6*)  =  e'(at). 

Note,  that  the  ‘empty  sum  equals  zero’,  and  ‘0°  =  1’  conventions  are  used. 

Proof: 

Except  for  the  explicit  form  given  for  the  subrisk,  the  proof  is  contained  in  the 
preceding  discussion.  Note  that  the  conditions  (Al),  (A2),  and  (T)  are  trivially 
satisfied  because  0  <  c(A*  —  A0)  <  oo.  Also  note  that  r*  <  oo  P„  —  a.s.,  V7r  €  [0,  l],  in 
view  of  Theorem  1.1.1.  As  for  the  subrisk,  in  view  of  Theorem  1.2.1  it  is  only  necessary 
to  show  that  it  satisfies  4.2.20  and  is  continuous  for  all  tv  <  b.  The  semicolons  are 
dropped  here  for  notational  convenience.  First  note  that  r(a)  =  R(a)  —  e(a),  due  to 
the  definition  of  K(a,b).  Next  note  that, 

<%)  =  (A1  -  (X1  -  A»  (1  -  1)  =  0,  We  [4, 

and  so,  5(7r)  =  e(7r)  on  this  set.  Hence,  the  boundary  conditions  of  4.2.20  have  been 
satisfied.  Also  observe  that, 

— 7r (1  —  7r)  d^n)  +  (u  +  7r)[d(K^  *  n)  —  d(7r)]  =  — c  V7r  G  (0, 1) 

where  the  derivative  is  taken  from  the  right,  as  usual.  Next,  one  can  show,  albeit 
tedious,  that, 

UD  =  UH  =  0.  (4.2.31) 

Thus  far,  have  argued  that  the  subrisk  satisfies  4.2.20.  It  only  remains  to.  demon¬ 
strate  the  continuity  on  7r  <  6.  Clearly,  iZ(7r)  is  continuous  for  all  it  except  possibly 
at  those  it  <  b  where, 


Nb(ir)  -  Nb( 7r  )  —  -1. 


(4.2.32) 
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It  is  not  too  difficult  to  see  that  Hi( 7r),  i  =  0, 1,  is  continuous  at  these  points.  On  the 
other  hand,  d(ir)  is  clearly  discontinuous  at  these  points.  A  simple  calculation  shows 
that, 

d{  7r)  —  d(TT~)  =  — C'(A1(l  —  7r)  +  A°7t), 

at  a  discontinuity.  On  the  other  hand  at  the  same  discontinuity  one  can  show  that, 

Di(n)  —  Di(n~)  =  C  i  =  0, 1. 

From  this  it  is  easy  to  see  that  D(tt)  is  also  discontinuous,  but  of  the  same  magnitude 
and  opposite  sign  as  d(?r).  Therefore,  the  sum  of  the  two  is  continuous  on  (0,  b).  The 
existence  and  uniqueness  of  the  solution  to  4.2.30  is  given  in  Appendix  II.  The  proof 
is  complete.  I 

Given  the  form  of  the  risk  in  the  theorem,  it  is  not  hard  to  see  why  explicit 
results  are  few  and  far  between  in  the  sequential  analysis  of  jump-type  processes, 
although  analogous  results  have  been  obtained  for  the  Wald  problem  dealing  with 
Poisson  processes  [DVORETZKY  &c]. 

Unlike  the  diffusion  formulation  in  section  4.1,  there  is  no  easy  extension  of  the 
time-homogeneous  result  to  the  most  general  case.  An  obvious  reason  is  that  in 
the  diffusion  case  the  drift  factors  out  of  the  characteristic  operator  (or  equivalently 
the  Dynkin  function  d(7r)),  whereas  in  the  Poisson  case  the  rates  do  not  factor  out 
completely  since  u  is  ‘tangled  up’  in  U.  As  a  result,  in  general  when  the  rates  are 
stochastic,  the  state  space  of  the  process  must  be  enlarged  to  include  the  rate.  This 
therefore  leads  to  a  partial  functional  differential  equation  for  the  risk  in  the  two 
variables,  7r,  and  u  (see  4.2.7).  In  addition,  since  the  7r-boundary  is  also  stochastic  in 
this  case,  one  is  forced  to  consider  the  [tt,  u)-boundary  of  the  (7rt,  Ut )  process.  This  has 
the  immediate  consequence  that  in  general,  one  must  extend  the  notion  of  threshold 
policies  from  intervals  to  include  open  sets. 

There  is  however  one  extension  that  can  be  obtained  from  the  result  above  with 
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little  difficulty.  This  is  the  case  when  nt  is  given  as  in  4.1.1  and  when  the  rate 
processes  {AJ  :  t  >  0}  obey, 

A*  =  A %,  t  >  0 ;  i  =  0, 1 ;  A1  >  A0.  (4.2.33) 

Obviously  (see  4.2.1),  {A*  :  t  >  0}  must  be  an  Jt-predictable  process  such  that 
At  >  0  Pit- a.s.  Under  these  assumptions  and  the  detectability  conditions  (Al)  and 
(A2)  one  obtains  the  following  theorem. 

Theorem  4.2.2  Assume  (Al)  and  (A2)  hold.  In  the  problem  of  Bayesian  sequential 
hypothesis  testing,  based  on  the  partial  observation  process  (see  4.2.1,  4.2.33), 

nt  =  f  (0A*  +  (1  -  0)A°)  ds  +  mt  t>  0, 

Jo 

with  running  cost, 

Ev[f  cads]  =  E«[[T  c{\\-\0a)ds]  c>0, 

Jo  Jo 

and  terminal  cost  e{i rT),  the  threshold  policy,  u*  =  ( rt,6 ,),  given  in  the  homogeneous 
case,  is  Bayesian.  Moreover,  the  risk  and  the  thresholds  are  exactly  the  same. 

Proof: 

First  note  that  (T)  follows  from  (Al).  The  computation  of  the  risk  is  given  by, 
p{n)  =  inf  E«[ 

t€T 

=  inf  Ek[  [T  c(Aj  -  A°)  ds  +  e(irT)]  (4.2.34) 

J0 

=  inf  E„ (—  [  (Aj  -  A°)Ur*(7Ts)  ds  +  e(irr)}, 
r€T  Jq 

with  r*  as  in  Theorem  4.2.1  and  U  the  operator  as  in  4.2.20.  Hence, 

p(tt)  =  r*(ir)  +  inf  E„[e(irT)  -  r*(7rT)j  =  r«(jr),  (4.2.35) 

r€T 

but  this  is  the  same  risk  as  in  the  homogeneous  case,  and  so  the  result  is  shown.  I 
This  concludes  the  section  and  chapter. 


f 

Jo 


cads  +  e(7Tr)] 
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APPENDIX  I 

Suppose  one  is  given  a  measurable  space  (0,  /)  upon  which  two  distinguished 
probability  measures  Pi,  i  =  0, 1,  are  defined.  Also  suppose  one  is  given  some  arbi¬ 
trary  (Pj-completed)  sub-cr-algebra  0{  C  7,  j  =  0, 1. 

Next  define  the  family  of  probability  measures,  {P„  :  0  <  tt  <  1},  via, 

P„{A)  =  ttP^A}  +  (1  -  7t)P0{A}  V  A  e  7,  tt  G  [0, 1]. 

Finally,  suppose  there  is  an  /-measurable  random  variable  6  :  fi  — ►  {0, 1}  such  that, 

Pn{9  =  l}  =  n  ,  P„{9  =  0}=1-tt,  (A.  1.1) 

i.e.,  6  =  1  with  prior  probability  it  and  vice  versa. 

Given'  the  above  set-up,  consider  first  the  problem  of  expressing  Ev[^\0t)  in 
terms  of  P^-conditional  expectations,  *  =  0, 1,  where  7  is  some  /-measurable  random 
variable  and  n  €  [0, 1].  Now  if  n  =  0, 1,  the  problem  is  trivial,  so  suppose  %  E  (0, 1). 
Noticing  that  Pi  <  P n  for  *  =  0, 1,  tt  €  (0,1),  it  follows  that  one  may  define  the 
Radon-Nikodym  derivatives, 

Ri(oj)  =  ~-{ w)  w  e  fi,  t  =  0, 1,  tt  e  (0, 1), 

and  then  there  holds, 

dP P 

E*[R'\0t]  =  =  { =  °> (AL2) 

where  P/,  i  =  0, 1,  P*  are  the  Oj-restrictions  of  the  respective  measures.  Given  this 
set-up,  one  can  prove  the  following  theorem. 

Theorem  A. 1.1  Let  n  G  [0,  l].  Then, 


Ev[n\ 0t]  =  TrRlExYi\Ot\  +  (1  -  n)R°Eo[^\0t\  P^-a.s.. 
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ndPj  +  {1  -  TT)dP0 
A 1  IP* 


Proof:  Let  A  6  Ot  and  compute, 

/  E^lOt)  dP„  =  f  ndP. „ 

JA  >/  A 

f  dPff  ,  _  . 

=  AWP’  {A-h3) 

-L 

using  the  Radon-Nikodym  theorem.  Continuing  from  the  last  line, 

[  Ev[^\0t\  dP ;  =  7T  [  ^dP-i  +  (1  -  7r)  [  7 dP0 
Ja  Ja  Ja 

=  tc  f  E1[^\Ot]dP1  +  {l-ir)  [  E0[i\Ot}dP0. 

Ja  Ja 

Now  for  *  =  0, 1  compute, 

^£,[710.]^  =  f^E*h\0t\  dP*  =  (A.  1.5) 

Combining  A.  1.2-5  yields, 

[  E^\0t\  dP„  =  w  [  E1[1\0t]Rl  dP„  +  (1  -  tt)  f  Eoh\Ot]R?  dP„, 

Ja  Ja  Ja 


(A.1.4) 


'A 

and  hence, 


JAE*h\0t]  dPn  =  JjnRlE^lOt]  +  (1  -  ir)R°tEQ[~t\Ot]}dPn. 

"Since  A  E  Ot  is  arbitrary,  it  follows  that, 

K[l\ 0t]  =  irR1tE1[1\0t]  +  (1  -  7r)Pt°P0[7|Ot], 


(A.1.6) 


except  on  0t  sets  of  P* -measure  zero.  I 

Now  suppose  that  one  takes  7  =  9 7',  9  the  binary  random  variable  defined  in 
the  beginning,  and  7'  any  /-measurable  random  variable.  Then  note, 

[  EtWlOt]  dPi  =  [  9i  dPi  =  [  7'  dPi  i  =  0, 1, 

J  A  Ja  J  An  {6=1} 
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and  hence, 

J^Ei[01'\Ot}dPi  =  {/Aydp.  { —  ! 

Thus,  for  this  choice  of  7  A.  1.6  becomes, 

^[^|0t]=rri2^i[y|0t].  (A.1.7) 


Now  A.  1.7  will  be  used  to  connect  the  a  posteriori  probability  process  to  the 
likelihood  ratio,  where  the  a  posteriori  probability  is  defined  as, 


tt*  =  P«{8  =  l|0t}  tt€[0,1], 


(A.  1.8) 


and  the  likelihood  ratio  is  given  by,* 

jpt  jp 

A,(w)  =  iit(")  =  Eo[^|0‘1  v"en,p0-o.s.,  (4.1.9) 

assuming  of  course  that  Pi  <P0-  Here,  {0t  •  t  >  0}  takes  on  the  role  of  a  filtration 
on  (0,  7),  either  right-continuous  or  discrete  (see  section  2.1).  Letting  7'  =  1  in  A.1.7 
one  immediately  obtains, 


nt  =  Pn{9  =  1| 0t)  =  E„[e\0t]  =  7 v-^  tv  G  (0, 1). 


(A.1.10) 


Using  the  Radon-Nikodym  Theorem  successively  and  observing  that  P{  <  P&*  one 
obtains  the  following  sequence  of  steps, 


7Tt  =  TV 


dP{ 


dp: 


dP, 


TvdPl  +  (1  -  Tv)dP$  . 


=  TV  - 


1  —  TV  -f  TV 

=  <A(A  t), 


dp\ 

dPS 


(A.1.11) 


where  the  last  line  serves  to  define  the  mapping  <j>  :  [0,  00]  — >  [0,1].  From  A.1.11 
one  obtains, 

Af  =  0-1(7rt)=  1--~7rl~7rt.  (A.1.12) 

TV  TVt 

*  The  symbol  can  be  interpreted  as  the  ratio  of  probabililty  masses  if  necessary. 


80 


Finally,  for  completeness  note  that  for  7r  =  1  that, 


7Tt  =  E\[6\0t]  =  7T  •  1  =  1, 


while  for  ir  =  0, 

7Tf  =  Eo[0\C>t]  =  7T  •  R\  —  0. 

From  A.1.11  and  A.1.12  it  is  clear  that  threshold  policies  based  upon  {7rt  :  t  >  0}  and 
{At  :  t  >  0}  are  equivalent  in  terms  of  optimality,  only  their  thresholds  differ. 
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APPENDIX  II 

In  this  appendix,  it  is  shown  that  there  exists  a  unique  solution  (a»,6*)  to  the 
system  of  equations, 


r(b  ;a,6)  =  e(6);  (A.2.1. a) 

r'(a;a,  b)  =  e'(a),  (A.2.1.6) 

with  r( 7r;  a,  b )  and  e(7r)  as  given  in  Theorem  4.2.1  (see  also  4.2.30).  First  suppose  that 
a  solution  (a*, 6*)  to  A.2.1  exists,  then  from  Theorem  1.2.1  and  its  corollary  it  follows 
that  r*( 7r)  =  r(7r;a*,6*)  E  C2(0,6*).  From  this  and  Theorem  3.2.1  it  follows  that 
r*  is  concave  on  (0,6*).  This  implies,  in  view  of  Theorem  3.2.2,  Lemma  4.2.1,  and 
A.2.1  that  the  conditions  (Cl),  (C2),  and  (C3)  hold.  Finally,  invoking  Theorem  3.1.2 
shows  that  the  solution  is  unique.  Thus,  it  is  only  necessary  to  show  that  at  least  one 
solution  to  A.2.1  exists. 

The  approach  to  accomplish  this  will  be  to  show  that  for  every  a  G  [0, 1]  there 
exists  a  6i(a)  E  [0, 1],  61(a)  >  a  such  that  A.2.1. a  becomes, 

r(6^(a);a, 61(a))  =  e(6x(a)).  (A.2.2) 

In  addition,  it  will  be  shown  that  the  mapping  a  i->  61(a)  is  continuous  and  onto  (0, 1). 
Following  that  it  will  be  demonstrated  using  A.2.1. b  that  there  exists  a  continuous 
mapping,  a  62(a),  satisfying, 

r'(a;a,  62(a))  =  e'(a),  (A.2.3) 

with  62(0)  =  1,  and  such  that  the  equation  62(a)  =  a  has  a  solution  for  0  <  a  <  1. 
Consequently,  the  two  curves  61(a), 62(a)  must  cross,  i.e.,  there  exists  an  a0  satisfying 
6 i(ao)  =  62(00).  Choosing  6*  =  6i(oq),  and  a*  =  ao  then  yields  a  pair  satisfying 


A.2.1. 
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To  begin,  from  Theorem  4.2.1  there  follows, 


r(7r;  a,  6)  =  d(ir,  b)  +  D(tt]  b)  +  K(a,  b)H(n;  b)  Vtt  <  b.  (A. 2.4) 


Letting  n  |  b  and  noting  iVj(6  )  =  0,  after  some  algebra  one  obtains, 

r(6“;  a,  b)  =  CfA^l  -  b)  +  A °b)  +  if(a,  6)(i^)u(l  -  b){\°  +  A1),  (A.2.5) 


where  by  definition  K(a,b )  is 


K(a,b) 


e(a)  —  [d(a;  b)  +  D(a ;  6)] 
H(a-,b) 


(A.2.6) 


In  the  proof  of  Theorem  4.2.1,  it  was  shown  that  H(ir,b)  and  [cf(7r;  6)  +  Z)(7r;6)]  are 
continuous  for  all  n  <  b.  Thus  K(a,b)  is  at  least  continuous  in  its  first  argument  and 
one  can  verify  albeit  tedious  that  the  mapping  a  b(a)  defined  implicitly  by  A.2.2 
and  A.2.5  is  continuous.  Now,  for  any  a  €  [0,  l],  suppose  b  =  &i(a)  is  chosen  so  that 
[S_16,  b)  3  a.  Then  from  A.2.6  there  follows, 


if  (a,  b ) 


e(a)-C(A1(l-a)  +  A°a)  /  a  \“ 
(1  —  a)(A°  +  A1)  \1 ^~a) 


{A.  2.7) 


After  some  simplification,  using  A.2.5-7  one  obtains, 


r(4-;  o,6)  =  C(A‘(1-  6)  +  \H)  +  («(o)  -  C(A‘  (1  -  a)  +  A°o)]  . 

(A.  2.8) 

From  A.2.8  it  is  clear  then  that  two  solutions  to  A.2.2  are  given  by  61  (0)  =  0,  and 
&i(l)  =  1.  Thus,  it  has  been  shown  that  a  &i(a)  is  continuous  and  onto  (0,  l). 

The  second  half  of  the  argument  deals  with  A.2.1.b.  First,  suppose  that  for  any 
a  e  [0, 1],  the  inequality  a  <  S-16  is  respected  by  b  =  b2(a).  Then  from  A.2.4,  one 
obtains  the  (right)  derivative, 


r'(Tr,a,b)  =  d'(n-,b)  +  D'(jr,b)  +  K(a,b)Ht(7r,b)  w  <  E  1b. 


{A.  2.9) 
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Recall  from  Theorem  4.2.1  that,  UD(n\b)  =  UH{n\b )  =  0,  and  so, 

a,b )  =  d'(7r;6)  +  _  &)  +  K{a,b)  b )  -  H(ir,  6)j  . 

(A.2.10) 

Substituting  tt  =  a  into  A.2.10  and  the  result  into  A.2.1.b  yields, 

D{Ea-,b)  -  D(a;b)  +  K{a,b)(^H(i:a]b)  -#(«;&))  =  (c»  -  d'{a;  6)). 

(A2.ll) 

The  last  line  can  be  simplified  in  two  ways.  First,  a  direct  computation  of  the  right 
derivative  d'{ir,  b)  yields, 

d'fcb)  =  ~C{\ 1  -  A°)(l  +  Nb( tt)).  (A2.12) 


Also,  from  A.2.2  and  A.2.5  there  holds, 


K{a,b) 


e(b)  —  C'(A1(1  —  6)  +  A°6)  /  b  \u 


(1-6)(A«  +  Ai) 


1-6 


(A2.13) 


Combining  A.2.11-13  with  a  little  algebra  yields  an  equation  whose  left-hand  side  is, 
(1  -  6)“+1  [J 9(Sa;  6)  -  D(a;  6)]  +  +  A°/j)  [5(Efl;  6)  -  JJ(a;  6)]  , 


(A2.14) 


and  whose  right-hand  side  is, 


-  6)*+I  +  -  A°)(l  -  4)*+‘(l  +  Nt («)).  (A2.15) 

Now  notice  that  (a,  6)  =  (0, 1)  solves  both  sides  (recall  SO  =  0).  The  only  troublesome 
term  is  in  A.2.15,  a(l  —  b)u+1Nb(a),  which  goes  to  zero  since, 

0  <  a(l  —  6)u+1iVfc(a)  <  a(l  —  6)'u+1(x(6)  —  x(a)) 


=  6(1  -  a)(l  -  6)“  -  a(l  -  b)u+1  ai-HU 


(A2.16) 
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Thus  far  have  shown  b2(0)  =  1,  and  from  A. 2. 10  it  is  clear  that  a  >->  b2(a )  is  con¬ 
tinuous,  arguing  as  before.  The  last  thing  to  show  is  that  there  exists  an  a 0  such 
that  0  <  62(a0)  =  oo  <  1,  i.e.,  that  the  curve  b2(a )  intercepts  the  line  b  =  a  in  the 
(a,  6)-plane.  For  if  this  is  shown,  the  two  curves  61(a)  and  b2(a)  must  cross,  yielding  a 
solution  to  the  problem.  To  show  b2(ao)  =  a 0  for  some  ao,  consider  A.2.4  for  all  6  in 
the  vicinity  of  a.  In  fact,  for  any  a  G  [0,  l],  choose  6  =  62(a)  to  satisfy  [£-16,6)  3  a , 
then  from  A.2.4 

r(tv;  a ,  6)  =  d(ir,  b)  +  K(a,  b)H(ir,  b ) 

(A.2.17) 


=  C'(A1(1  -  tt)  +  A0tt)  +  K{a,b)  (^~~j  (1  -  Jr)(A°  +  A1), 


and  so, 


r'(7r;a,b)  =  -C^A1  -  A0)  -  K(a,b)  [\°  +  X1).  (A.2.18) 

Substituting  for  K(a,b)  from  A.2.13  under  the  condition  a  G  [£-16, 6)  yields, 

r '(*;«, 6)  =  -C(Xl  -  \°)  -  e,W-C(A1(l-t)  +  ^)  t 

1-6  \(l-b)vrj  \  n  J 


Thus,  setting  r'(a;a,  6)  =  e'(a)  and  using  A.2.19  one  gets, 


-a{  1  -  6)C(A1  -  A0)  -  («  +  a)  fe(6)  -  C(Ax(l  -  6)  +  A°6) 


Kl  ~a)Y 

(1  —  6)  a 


(A.2.19) 


=  a(l  —  b)e'(a). 


(A.  2.20) 

Evaluating  A.2.20  on  the  line  6  =  a  gives, 

—o(l  -  a)C(Ax  -  A0)  -  («  +  a)  [«(«)  -  C^A1  (1  -  a)  +  A0 a)]  =  o(l  -  a)e'(a).  (A.2.21) 
After  some  algebraic  simplification  A.2.21  can  be  rewritten, 


a(u  +  1)  -  c  =  0,  (A.2.22) 

i.e.,  a  =  Al — L.  So  if  c  <  u  +  1,  then  a  solution  to  A.2.1.b  exists  on  the  line  a  =  6. 
In  this  case,  there  exists  a  solution  (a*, 6*)  solving  A.2.1,  as  argued  previously.  Notice 
that  the  condition  c  <  u  +  1  holds  automatically  for  all  c  <  1.  On  the  other  hand, 
this  condition  can  also  always  be  obtained  by  the  simple  change  of  time  scale.  This 
completes  the  argument. 
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